Data_Exploration_for_Tuning_a_Foundation_Model_.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Eddiebee/AI-Craft/blob/main/Data_Exploration_for_Tuning_a_Foundation_Model_.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "c25470fb-da7a-4bab-b1fa-c5e54c6ab96a",
      "metadata": {
        "tags": [],
        "id": "c25470fb-da7a-4bab-b1fa-c5e54c6ab96a"
      },
      "source": [
        "# L2: Data Exploration for Tuning a Foundation Model"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9eee94e3-0a12-4616-8a5b-cd0c78f88982",
      "metadata": {
        "tags": [],
        "id": "9eee94e3-0a12-4616-8a5b-cd0c78f88982"
      },
      "source": [
        "**Project environment setup:**\n",
        "\n",
        "- Load credentials and relevant Python Libraries\n",
        "- If you were running this notebook locally, you would first install Vertex AI. In this classroom, this is already installed.\n",
        "\n",
        "```\n",
        "!pip install google-cloud-aiplatform\n",
        "```\n",
        "- You can download the `requirements.txt` for this course from the workspace of this lab. `File --> Open...`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d8d58769-1fea-4c2e-8ce1-86a8489a6e41",
      "metadata": {
        "height": 48,
        "tags": [],
        "id": "d8d58769-1fea-4c2e-8ce1-86a8489a6e41"
      },
      "outputs": [],
      "source": [
        "from utils import authenticate\n",
        "credentials, PROJECT_ID = authenticate()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f53c0073-9723-40cd-8205-61a50729f5f6",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "f53c0073-9723-40cd-8205-61a50729f5f6"
      },
      "outputs": [],
      "source": [
        "REGION = \"us-central1\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9985efad-4a9e-4fd0-b35e-569d17d8cf05",
      "metadata": {
        "tags": [],
        "id": "9985efad-4a9e-4fd0-b35e-569d17d8cf05"
      },
      "source": [
        "- Import the [Vertex AI](https://cloud.google.com/vertex-ai) SDK.\n",
        "- The library helps to interact with the Vertex AI services in the cloud.\n",
        "- Initialize it."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cfbb69b4-7401-4f4e-9067-86e6f5684a03",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "cfbb69b4-7401-4f4e-9067-86e6f5684a03"
      },
      "outputs": [],
      "source": [
        "import vertexai"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7802ff08-ac5a-4252-a464-237ab2f86f5a",
      "metadata": {
        "height": 65,
        "id": "7802ff08-ac5a-4252-a464-237ab2f86f5a"
      },
      "outputs": [],
      "source": [
        "vertexai.init(project = PROJECT_ID,\n",
        "              location = REGION,\n",
        "              credentials = credentials)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "20ec9443-b52d-4157-8b03-44774561c375",
      "metadata": {
        "tags": [],
        "id": "20ec9443-b52d-4157-8b03-44774561c375"
      },
      "source": [
        "- Import [BigQuery](https://cloud.google.com/bigquery) to use as your data warehouse.\n",
        "- Initialize the client to start interacting with the data warehouse, send SQL and retrieve data into the notebook."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "e48d8307-09a5-4518-89ea-6d46cd41353f",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "e48d8307-09a5-4518-89ea-6d46cd41353f"
      },
      "outputs": [],
      "source": [
        "from google.cloud import bigquery"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "a9a99c48-ce21-4628-9bf6-ec16aeea9e51",
      "metadata": {
        "height": 48,
        "id": "a9a99c48-ce21-4628-9bf6-ec16aeea9e51"
      },
      "outputs": [],
      "source": [
        "bq_client = bigquery.Client(project=PROJECT_ID,\n",
        "                            credentials = credentials)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "33cf86c4-e2e5-416f-ab57-04f53cbfa703",
      "metadata": {
        "id": "33cf86c4-e2e5-416f-ab57-04f53cbfa703"
      },
      "source": [
        "## Stack Overflow Public Dataset"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "44d8c159-5b14-40f1-a657-fdd451a82ddf",
      "metadata": {
        "id": "44d8c159-5b14-40f1-a657-fdd451a82ddf"
      },
      "source": [
        "- You will use [Stack Overflow Data](https://cloud.google.com/blog/topics/public-datasets/google-bigquery-public-datasets-now-include-stack-overflow-q-a) on BigQuery Public Datasets.\n",
        "- The datasets include questions, answers and metadata related to Stack Overflow questions. Within this dataset, there are tables with data.\n",
        "- Create a SQL query."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "27a3e57a-8bea-44bb-891d-951d33997be9",
      "metadata": {
        "height": 116,
        "id": "27a3e57a-8bea-44bb-891d-951d33997be9"
      },
      "outputs": [],
      "source": [
        "QUERY_TABLES = \"\"\"\n",
        "SELECT\n",
        "  table_name\n",
        "FROM\n",
        "  `bigquery-public-data.stackoverflow.INFORMATION_SCHEMA.TABLES`\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "0597e2d5-959e-475e-8434-b9da91c84739",
      "metadata": {
        "id": "0597e2d5-959e-475e-8434-b9da91c84739"
      },
      "source": [
        "- The query is asking to retrieve `table_name` of all the `TABLES`\n",
        "- Use the client to send your SQL and retrieve the data (tables names)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ca80b70a-c989-4448-b9bf-bafbadef0d17",
      "metadata": {
        "height": 31,
        "id": "ca80b70a-c989-4448-b9bf-bafbadef0d17"
      },
      "outputs": [],
      "source": [
        "query_job = bq_client.query(QUERY_TABLES)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "9b3bccd0-7f04-4864-901b-402e505f341b",
      "metadata": {
        "height": 65,
        "id": "9b3bccd0-7f04-4864-901b-402e505f341b",
        "outputId": "5c43249b-2c9a-4c7d-9165-648a71691558"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "posts_answers\n",
            "users\n",
            "posts_orphaned_tag_wiki\n",
            "posts_tag_wiki\n",
            "stackoverflow_posts\n",
            "posts_questions\n",
            "comments\n",
            "posts_tag_wiki_excerpt\n",
            "posts_wiki_placeholder\n",
            "posts_privilege_wiki\n",
            "post_history\n",
            "badges\n",
            "post_links\n",
            "tags\n",
            "votes\n",
            "posts_moderator_nomination\n"
          ]
        }
      ],
      "source": [
        "for row in query_job:\n",
        "    for value in row.values():\n",
        "        print(value)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "861de62b-a94d-4b59-98fc-f7f027b0b13c",
      "metadata": {
        "id": "861de62b-a94d-4b59-98fc-f7f027b0b13c"
      },
      "source": [
        "## Data Retrieval"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "a4111b8f-c9ab-4b47-9a85-3430650c375c",
      "metadata": {
        "id": "a4111b8f-c9ab-4b47-9a85-3430650c375c"
      },
      "source": [
        "- You'll fetch some data from the data warehouse and store it in Pandas dataframe for visualization.\n",
        "- Select all columns from  `posts_questions` and put the `LIMIT` as 3."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "2ffec7eb-7ba4-472e-85ba-39fce6ab706f",
      "metadata": {
        "height": 133,
        "id": "2ffec7eb-7ba4-472e-85ba-39fce6ab706f"
      },
      "outputs": [],
      "source": [
        "INSPECT_QUERY = \"\"\"\n",
        "SELECT\n",
        "    *\n",
        "FROM\n",
        "    `bigquery-public-data.stackoverflow.posts_questions`\n",
        "LIMIT 3\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ccbe4aca-8d5b-4e3f-8edd-41e32fca49bf",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "ccbe4aca-8d5b-4e3f-8edd-41e32fca49bf"
      },
      "outputs": [],
      "source": [
        "import pandas as pd"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "94d974a6-d922-46ca-89fb-c29abbed0128",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "94d974a6-d922-46ca-89fb-c29abbed0128"
      },
      "outputs": [],
      "source": [
        "query_job = bq_client.query(INSPECT_QUERY)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d6bd733b-e0d9-452c-a2d9-ad4072e51c78",
      "metadata": {
        "tags": [],
        "id": "d6bd733b-e0d9-452c-a2d9-ad4072e51c78"
      },
      "source": [
        "- Take the results of the query `-->` create an arrow table (which is part of [Apache Framework](https://arrow.apache.org/docs/index.html)) `-->` which goes into a Pandas dataframe.\n",
        "- This allows for data to be in a format which is easier to read and explore with Pandas."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "491d400f-a9a4-45bf-92f7-666e2ca6bda4",
      "metadata": {
        "height": 99,
        "tags": [],
        "id": "491d400f-a9a4-45bf-92f7-666e2ca6bda4",
        "outputId": "0406dd53-7224-4411-e527-c184dcb3e3a5"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>id</th>\n",
              "      <th>title</th>\n",
              "      <th>body</th>\n",
              "      <th>accepted_answer_id</th>\n",
              "      <th>answer_count</th>\n",
              "      <th>comment_count</th>\n",
              "      <th>community_owned_date</th>\n",
              "      <th>creation_date</th>\n",
              "      <th>favorite_count</th>\n",
              "      <th>last_activity_date</th>\n",
              "      <th>last_edit_date</th>\n",
              "      <th>last_editor_display_name</th>\n",
              "      <th>last_editor_user_id</th>\n",
              "      <th>owner_display_name</th>\n",
              "      <th>owner_user_id</th>\n",
              "      <th>parent_id</th>\n",
              "      <th>post_type_id</th>\n",
              "      <th>score</th>\n",
              "      <th>tags</th>\n",
              "      <th>view_count</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>320268</td>\n",
              "      <td>Html.ActionLink doesn’t render # properly</td>\n",
              "      <td>&lt;p&gt;When using Html.ActionLink passing a string...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>None</td>\n",
              "      <td>2008-11-26T10:42:37.477Z</td>\n",
              "      <td>0</td>\n",
              "      <td>2009-02-06T20:13:54.370Z</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Paulo</td>\n",
              "      <td>NaN</td>\n",
              "      <td>None</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>asp.net-mvc</td>\n",
              "      <td>390</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>324003</td>\n",
              "      <td>Primitive recursion</td>\n",
              "      <td>&lt;p&gt;how  will i define the function 'simplify' ...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>None</td>\n",
              "      <td>2008-11-27T15:12:37.497Z</td>\n",
              "      <td>0</td>\n",
              "      <td>2012-09-25T19:54:40.597Z</td>\n",
              "      <td>2012-09-25T19:54:40.597Z</td>\n",
              "      <td>Marcin</td>\n",
              "      <td>1288.0</td>\n",
              "      <td>None</td>\n",
              "      <td>41000.0</td>\n",
              "      <td>None</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>haskell|lambda|functional-programming|lambda-c...</td>\n",
              "      <td>497</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>390605</td>\n",
              "      <td>While vs. Do While</td>\n",
              "      <td>&lt;p&gt;I've seen both the blocks of code in use se...</td>\n",
              "      <td>390608.0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>None</td>\n",
              "      <td>2008-12-24T01:49:54.230Z</td>\n",
              "      <td>2</td>\n",
              "      <td>2008-12-24T03:08:55.897Z</td>\n",
              "      <td>None</td>\n",
              "      <td>None</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Unkwntech</td>\n",
              "      <td>115.0</td>\n",
              "      <td>None</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>language-agnostic|loops</td>\n",
              "      <td>11262</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "       id                                      title  \\\n",
              "0  320268  Html.ActionLink doesn’t render # properly   \n",
              "1  324003                        Primitive recursion   \n",
              "2  390605                         While vs. Do While   \n",
              "\n",
              "                                                body  accepted_answer_id  \\\n",
              "0  <p>When using Html.ActionLink passing a string...                 NaN   \n",
              "1  <p>how  will i define the function 'simplify' ...                 NaN   \n",
              "2  <p>I've seen both the blocks of code in use se...            390608.0   \n",
              "\n",
              "   answer_count  comment_count community_owned_date             creation_date  \\\n",
              "0             0              0                 None  2008-11-26T10:42:37.477Z   \n",
              "1             0              0                 None  2008-11-27T15:12:37.497Z   \n",
              "2             0              0                 None  2008-12-24T01:49:54.230Z   \n",
              "\n",
              "   favorite_count        last_activity_date            last_edit_date  \\\n",
              "0               0  2009-02-06T20:13:54.370Z                      None   \n",
              "1               0  2012-09-25T19:54:40.597Z  2012-09-25T19:54:40.597Z   \n",
              "2               2  2008-12-24T03:08:55.897Z                      None   \n",
              "\n",
              "  last_editor_display_name  last_editor_user_id owner_display_name  \\\n",
              "0                     None                  NaN              Paulo   \n",
              "1                   Marcin               1288.0               None   \n",
              "2                     None                  NaN          Unkwntech   \n",
              "\n",
              "   owner_user_id parent_id  post_type_id  score  \\\n",
              "0            NaN      None             1      0   \n",
              "1        41000.0      None             1      0   \n",
              "2          115.0      None             1      0   \n",
              "\n",
              "                                                tags  view_count  \n",
              "0                                        asp.net-mvc         390  \n",
              "1  haskell|lambda|functional-programming|lambda-c...         497  \n",
              "2                            language-agnostic|loops       11262  "
            ]
          },
          "execution_count": 13,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "stack_overflow_df = query_job\\\n",
        "    .result()\\\n",
        "    .to_arrow()\\\n",
        "    .to_pandas()\n",
        "stack_overflow_df.head()"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "59d1fac8-8e5e-4986-ae30-9e374ff5078b",
      "metadata": {
        "id": "59d1fac8-8e5e-4986-ae30-9e374ff5078b"
      },
      "source": [
        "### Dealing with Large Datasets\n",
        "\n",
        "- Large datasets for LLMs often don't fit into memory.\n",
        "- Select all of the columns and rows of the table `posts_questions`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "863e9ce6-206b-4032-8eb8-9e6204d8b158",
      "metadata": {
        "height": 116,
        "tags": [],
        "id": "863e9ce6-206b-4032-8eb8-9e6204d8b158"
      },
      "outputs": [],
      "source": [
        "QUERY_ALL = \"\"\"\n",
        "SELECT\n",
        "    *\n",
        "FROM\n",
        "    `bigquery-public-data.stackoverflow.posts_questions` q\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "632b3e00-5014-48f3-860c-895ed7b1e263",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "632b3e00-5014-48f3-860c-895ed7b1e263"
      },
      "outputs": [],
      "source": [
        "query_job = bq_client.query(QUERY_ALL)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ca55b26f-1e9c-46f2-933b-0d2e85269948",
      "metadata": {
        "height": 133,
        "tags": [],
        "id": "ca55b26f-1e9c-46f2-933b-0d2e85269948",
        "outputId": "a7b675e7-5a0f-4114-95ce-ff14393339ee"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The DataFrame is too large to load into memory. 403 Response too large to return. Consider specifying a destination table in your job configuration. For more details, see https://cloud.google.com/bigquery/troubleshooting-errors\n",
            "\n",
            "Location: US\n",
            "Job ID: e6660106-e9d7-4464-9c2a-f575495c0d3d\n",
            "\n"
          ]
        }
      ],
      "source": [
        "try:\n",
        "    stack_overflow_df = query_job\\\n",
        "    .result()\\\n",
        "    .to_arrow()\\\n",
        "    .to_pandas()\n",
        "except Exception as e:\n",
        "    print('The DataFrame is too large to load into memory.', e)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "9999ff41-e115-4491-84c3-9d7f04fafd85",
      "metadata": {
        "tags": [],
        "id": "9999ff41-e115-4491-84c3-9d7f04fafd85"
      },
      "source": [
        "**Note:** The data is too large to return, as it is not fitting into memory."
      ]
    },
    {
      "cell_type": "markdown",
      "id": "d406553f-c356-4dc8-a119-7d8f02787d4b",
      "metadata": {
        "id": "d406553f-c356-4dc8-a119-7d8f02787d4b"
      },
      "source": [
        "#### Joining Tables and Query Optimization\n",
        "\n",
        "- When working with (large) data, query optimizing is needed in order to save time and resources.\n",
        "- Select questions as `input_text` (column 1), answers as `output_text` (column 2).\n",
        "- Take the questions from `posts_questions` and answers from `posts_answers`.\n",
        "- Join the questions and their corresponding accepted answers based on their same `unique ID`.\n",
        "- Making sure the question is about `Python`, and that it `has an answer`. And the date the question was posted is on or after `2020-01-01`\n",
        "- Limit as 10,000"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "71dc0eb0-ca3d-4d29-ae7e-f3b40cab9184",
      "metadata": {
        "height": 303,
        "tags": [],
        "id": "71dc0eb0-ca3d-4d29-ae7e-f3b40cab9184"
      },
      "outputs": [],
      "source": [
        "QUERY = \"\"\"\n",
        "SELECT\n",
        "    CONCAT(q.title, q.body) as input_text,\n",
        "    a.body AS output_text\n",
        "FROM\n",
        "    `bigquery-public-data.stackoverflow.posts_questions` q\n",
        "JOIN\n",
        "    `bigquery-public-data.stackoverflow.posts_answers` a\n",
        "ON\n",
        "    q.accepted_answer_id = a.id\n",
        "WHERE\n",
        "    q.accepted_answer_id IS NOT NULL AND\n",
        "    REGEXP_CONTAINS(q.tags, \"python\") AND\n",
        "    a.creation_date >= \"2020-01-01\"\n",
        "LIMIT\n",
        "    10000\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "8affe03b-6be6-4be0-ad09-703f47dc31b9",
      "metadata": {
        "height": 31,
        "tags": [],
        "id": "8affe03b-6be6-4be0-ad09-703f47dc31b9"
      },
      "outputs": [],
      "source": [
        "query_job = bq_client.query(QUERY)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "d23c0df2-2365-4a49-9dee-f49c800312dd",
      "metadata": {
        "height": 116,
        "tags": [],
        "id": "d23c0df2-2365-4a49-9dee-f49c800312dd",
        "outputId": "37c3a8c7-fb84-41d6-c1e2-02cacef1a087"
      },
      "outputs": [
        {
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>input_text</th>\n",
              "      <th>output_text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Sort aggregated column in multiindex dataframe...</td>\n",
              "      <td>&lt;p&gt;In last pandas versions is possible sorting...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>How to decode LabelEncoder implemented column ...</td>\n",
              "      <td>&lt;p&gt;You need one encoder per column - you canno...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                                          input_text  \\\n",
              "0  Sort aggregated column in multiindex dataframe...   \n",
              "1  How to decode LabelEncoder implemented column ...   \n",
              "\n",
              "                                         output_text  \n",
              "0  <p>In last pandas versions is possible sorting...  \n",
              "1  <p>You need one encoder per column - you canno...  "
            ]
          },
          "execution_count": 19,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "### this may take some seconds to run\n",
        "stack_overflow_df = query_job.result()\\\n",
        "                        .to_arrow()\\\n",
        "                        .to_pandas()\n",
        "\n",
        "stack_overflow_df.head(2)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "05bdc676-a15d-4d28-a95c-7b451c6427b7",
      "metadata": {
        "tags": [],
        "id": "05bdc676-a15d-4d28-a95c-7b451c6427b7"
      },
      "source": [
        "### Adding Instructions\n",
        "\n",
        "- Instructions for LLMs have been shown to improve\n",
        "model performance and generalization to unseen tasks [(Google, 2022)](https://arxiv.org/pdf/2210.11416.pdf).\n",
        "- Wihtout the instruction, it is only question and answer. Model might not understand what to do.\n",
        "- With the instructions, the model gets a guideline as to what task to perform."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "66208ed7-7325-4613-8b57-fedfa9c85aef",
      "metadata": {
        "height": 116,
        "tags": [],
        "id": "66208ed7-7325-4613-8b57-fedfa9c85aef"
      },
      "outputs": [],
      "source": [
        "INSTRUCTION_TEMPLATE = f\"\"\"\\\n",
        "Please answer the following Stackoverflow question on Python. \\\n",
        "Answer it like you are a developer answering Stackoverflow questions.\n",
        "\n",
        "Stackoverflow question:\n",
        "\"\"\""
      ]
    },
    {
      "cell_type": "markdown",
      "id": "645069aa-7e45-4fa3-91e4-665b9f7c17a4",
      "metadata": {
        "tags": [],
        "id": "645069aa-7e45-4fa3-91e4-665b9f7c17a4"
      },
      "source": [
        "- A new column will combine `INSTRUCTION_TEMPLATE` and the question `input_text`.\n",
        "- This avoids overwritting of any existing column which might be needed."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "ed30a78b-6b4a-4b07-b66f-b5147e6ec062",
      "metadata": {
        "height": 48,
        "tags": [],
        "id": "ed30a78b-6b4a-4b07-b66f-b5147e6ec062"
      },
      "outputs": [],
      "source": [
        "stack_overflow_df['input_text_instruct'] = INSTRUCTION_TEMPLATE + ' '\\\n",
        "    + stack_overflow_df['input_text']"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "007f22f9-cb41-4035-842e-bd6b27c01bf4",
      "metadata": {
        "id": "007f22f9-cb41-4035-842e-bd6b27c01bf4"
      },
      "source": [
        "### Dataset for Tuning\n",
        "\n",
        "- Divide the data into a training and evaluation. By default, 80/20 split is used.\n",
        "- This (80/20 split) allows for more data to be used for tuning. The evaluation split is used as unseen data during tuning to evaluate performance.\n",
        "- The `random_state` parameter is used to ensure random sampling for a fair comparison."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7dc2940f-735a-4fe0-b579-b8e63d161d57",
      "metadata": {
        "height": 31,
        "id": "7dc2940f-735a-4fe0-b579-b8e63d161d57"
      },
      "outputs": [],
      "source": [
        "from sklearn.model_selection import train_test_split"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "b6c97888-7eb8-49f0-b3e7-2a417d931b49",
      "metadata": {
        "height": 133,
        "id": "b6c97888-7eb8-49f0-b3e7-2a417d931b49"
      },
      "outputs": [],
      "source": [
        "train, evaluation = train_test_split(\n",
        "    stack_overflow_df,\n",
        "    ### test_size=0.2 means 20% for evaluation\n",
        "    ### which then makes train set to be of 80%\n",
        "    test_size=0.2,\n",
        "    random_state=42\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "1cb841d8-e71a-464c-8cb3-3c712a16dc6a",
      "metadata": {
        "id": "1cb841d8-e71a-464c-8cb3-3c712a16dc6a"
      },
      "source": [
        "#### Different Datasets and Flow\n",
        "\n",
        "- Versioning data is important.\n",
        "- It allows for reproducibility, traceability, and maintainability of machine learning models.\n",
        "- Get the timestamp."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "f0b0f876-10bf-4dbf-ae0f-1f90d62cb72c",
      "metadata": {
        "height": 31,
        "id": "f0b0f876-10bf-4dbf-ae0f-1f90d62cb72c"
      },
      "outputs": [],
      "source": [
        "import datetime"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "cf919020-de23-4e02-895f-3cf22fe75f91",
      "metadata": {
        "height": 31,
        "id": "cf919020-de23-4e02-895f-3cf22fe75f91"
      },
      "outputs": [],
      "source": [
        "date = datetime.datetime.now().strftime(\"%H:%d:%m:%Y\")"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "5989a1ea-1bdb-4e43-a4c1-fec76812c26d",
      "metadata": {
        "id": "5989a1ea-1bdb-4e43-a4c1-fec76812c26d"
      },
      "source": [
        "- Generate a `jsonl` file.\n",
        "- Name it as `tune_data_stack_overflow_python_qa-{date}`"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7427ff3f-6f3a-4053-982c-3aa1fcfb656a",
      "metadata": {
        "height": 48,
        "id": "7427ff3f-6f3a-4053-982c-3aa1fcfb656a"
      },
      "outputs": [],
      "source": [
        "cols = ['input_text_instruct','output_text']\n",
        "tune_jsonl = train[cols].to_json(orient=\"records\", lines=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "6727f808-a216-45e3-8073-4545e23c796d",
      "metadata": {
        "height": 48,
        "id": "6727f808-a216-45e3-8073-4545e23c796d"
      },
      "outputs": [],
      "source": [
        "training_data_filename = f\"tune_data_stack_overflow_\\\n",
        "                            python_qa-{date}.jsonl\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "7f639359-f873-404f-9fd7-bf88db8c52a0",
      "metadata": {
        "height": 48,
        "tags": [],
        "id": "7f639359-f873-404f-9fd7-bf88db8c52a0"
      },
      "outputs": [],
      "source": [
        "with open(training_data_filename, \"w\") as f:\n",
        "    f.write(tune_jsonl)"
      ]
    },
    {
      "cell_type": "markdown",
      "id": "44daae92-2502-4913-a53f-7dee952797fe",
      "metadata": {
        "id": "44daae92-2502-4913-a53f-7dee952797fe"
      },
      "source": [
        "## Try it Yourself! - Evaluation Set\n",
        "\n",
        "The code above generted a `jsonl` file for the `train` set. Now, its time for you to make the `evaluation` set, which you can name as `tune_eval_data_stack_overflow_python_qa-{date}.jsonl`. The code for that is also provided to you in the drop down below, but we encourage you to try it yourself first before you look at it.\n",
        "\n",
        "<details>\n",
        "  <summary><font size=\"2\" color=\"darkgreen\"><b>Code for Evaluation Set (Click to expand)</b></font></summary>\n",
        "    \n",
        "```python\n",
        "\n",
        "cols = ['input_text_instruct','output_text']\n",
        "### you need to use the \"evaluation\" set now\n",
        "tune_jsonl = evaluation[cols].to_json(orient=\"records\", lines=True)\n",
        "\n",
        "### change the file name\n",
        "### use \"tune_eval_data_stack_overflow_python_qa-{date}.jsonl\"\n",
        "evaluation_data_filename = f\"tune_eval_data_stack_overflow_\\\n",
        "                            python_qa-{date}.jsonl\"\n",
        "\n",
        "### write the file\n",
        "with open(evaluation_data_filename, \"w\") as f:\n",
        "    f.write(tune_jsonl)\n",
        "\n",
        "```"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "92813bc1-1a68-473d-be14-d5ee480faedc",
      "metadata": {
        "height": 48,
        "id": "92813bc1-1a68-473d-be14-d5ee480faedc"
      },
      "outputs": [],
      "source": [
        "eval_jsonl = evaluation[cols].to_json(orient=\"records\",\n",
        "                                      lines=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "738b9e97-5f4f-49ae-ba3f-d130dbb6674b",
      "metadata": {
        "height": 48,
        "id": "738b9e97-5f4f-49ae-ba3f-d130dbb6674b"
      },
      "outputs": [],
      "source": [
        "eval_training_data_filename = f\"tune_eval_data_stack_overflow_\\\n",
        "                                python_qa-{date}.jsonl\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "id": "38af5411",
      "metadata": {
        "height": 48,
        "id": "38af5411"
      },
      "outputs": [],
      "source": [
        "with open(eval_training_data_filename, \"w\") as f:\n",
        "    f.write(eval_jsonl)"
      ]
    }
  ],
  "metadata": {
    "environment": {
      "kernel": "python3",
      "name": "tf2-cpu.2-11.m113",
      "type": "gcloud",
      "uri": "gcr.io/deeplearning-platform-release/tf2-cpu.2-11:m113"
    },
    "kernelspec": {
      "display_name": "Python 3 (ipykernel)",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.10.13"
    },
    "colab": {
      "provenance": [],
      "include_colab_link": true
    }
  },
  "nbformat": 4,
  "nbformat_minor": 5
}