code_examples/openai/fine_tuning.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0677690b-bc59-41b1-bf6c-1340b5f6e4a8",
   "metadata": {},
   "source": [
    "# OpenAI - fine-tuning example\n",
    "\n",
    "The guide is a companion to the paper *\"Generative LLMs and Textual Analysis in Accounting:(Chat)GPT as Research Assistant?\"* ([SSRN](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4429658))\n",
    "\n",
    "**Author:** [Ties de Kok](https://www.tiesdekok.com)    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "799fdde1-c830-4edd-a062-56592955a0cf",
   "metadata": {},
   "source": [
    "----\n",
    "# Imports\n",
    "----\n",
    "\n",
    "\n",
    "All the dependencies required for this notebook are provided in the `environment.yml` file.\n",
    "\n",
    "To install: `conda env create -f environment.yml` --> this creates the `gllm` environment.\n",
    "\n",
    "I recommend using Python 3.9 or higher to avoid dependency conflicts."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1969ecc-dd72-475a-ae51-79bdc83c84d7",
   "metadata": {},
   "source": [
    "**Python built-in libraries**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a4ec5462-905d-4b7c-b748-a43dbbe84b7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os, sys, re, copy, random, json, time, datetime\n",
    "from pathlib import Path\n",
    "import getpass"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97ae5855-5a12-4098-9545-495fbc540506",
   "metadata": {},
   "source": [
    "**Libraries for interacting with the OpenAI API**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c77af21a-0a1c-4610-a213-a4565742bbaa",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "import openai\n",
    "import tiktoken"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ccebae9d-fd4c-42da-81ea-7b4672bbec07",
   "metadata": {},
   "source": [
    "**General helper libraries**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "c877637b-27c2-4d17-934d-a1e0c65422fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from tqdm.notebook import tqdm"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c95257da-1df7-44de-8cc9-49071d6db842",
   "metadata": {},
   "source": [
    "### Settings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f0ccdeb1-e3f2-4f96-b924-9a36287e78ed",
   "metadata": {},
   "outputs": [],
   "source": [
    "pd.options.mode.chained_assignment = None  # default='warn'\n",
    "pd.set_option('display.max_columns', 150)\n",
    "pd.set_option('display.max_rows', 150)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4a0b98b8-afcd-40b6-a2cb-173f95a14e98",
   "metadata": {},
   "source": [
    "### Utility functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "da331675-1d54-41a2-9c04-c77d331c3c70",
   "metadata": {},
   "outputs": [],
   "source": [
    "## This function makes it easier to print rendered markdown through a code cell.\n",
    "\n",
    "from IPython.display import Markdown\n",
    "\n",
    "def mprint(text, *args, **kwargs):\n",
    "    if 'end' in kwargs.keys():\n",
    "        text += kwargs['end']\n",
    "        \n",
    "    display(Markdown(text))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9fa6682f-faa4-4b4c-9b39-bcb383aede60",
   "metadata": {},
   "source": [
    "---\n",
    "## Set up OpenAI\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5853aba4-baea-4589-b18c-e64427f5c0f1",
   "metadata": {},
   "source": [
    "To use the OpenAI API you will need an API key. If you don't have one, follow these steps:   \n",
    "\n",
    "1. Create an OpenAI account --> https://platform.openai.com   \n",
    "2. Create an OpenAI API key --> https://platform.openai.com/account/api-keys   \n",
    "3. You will get \\\\$5 in free credits if you create a new account. If you've used that up, you will need to add a payment method. The code in this notebook will cost less than a dollar to run.\n",
    "\n",
    "Once you have your OpenAI Key, you can set it as the `OPENAI_API_KEY` environment variable (recommended) or enter it directly below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "b60c5fef-cc7c-4eaf-9e94-5ddaf999a27e",
   "metadata": {},
   "outputs": [],
   "source": [
    "if 'OPENAI_API_KEY' not in os.environ:\n",
    "    os.environ['OPENAI_API_KEY'] = getpass.getpass(prompt='Enter your API key: ')\n",
    "    \n",
    "openai.api_key = os.environ['OPENAI_API_KEY']\n",
    "    \n",
    "## KEEP YOUR KEY SECURE, ANYONE WITH ACCESS TO IT CAN GENERATE COSTS ON YOUR ACCOUNT!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8253090c-69ba-4b5a-ab53-5c6119db5855",
   "metadata": {},
   "source": [
    "----\n",
    "# Create demo dataset\n",
    "---\n",
    "\n",
    "Let's say our task is to find all words in a sentence that are wrapped by <>. For example:\n",
    "\n",
    "`This is a <word> and this is another <thing> plus some other text.`\n",
    "\n",
    "We want: \n",
    "- \"word\"\n",
    "- \"thing\"\n",
    "\n",
    "*Note:* this is a task easily solved using regular expressions, so it makes for an easy example as we can create as much training data as we want. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9856d1dc-0248-42fa-b6ed-79c2d0e07068",
   "metadata": {},
   "source": [
    "### Load some statements "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "0fac213f-9c29-40d2-ac11-7b69b694e628",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open(Path.cwd() / \"data\" / \"statements.json\", \"r\", encoding = \"utf-8\") as f:\n",
    "    statement_list = json.load(f)\n",
    "\n",
    "statement_df = pd.DataFrame(statement_list)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa5bdb6a-5d7a-4867-9487-95cfbd9577b0",
   "metadata": {},
   "source": [
    "### Randomly add <>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "79a77aa6-1720-44d1-bb49-6d8dd971b043",
   "metadata": {},
   "outputs": [],
   "source": [
    "all_sentences = []\n",
    "for statement in statement_df.statement:\n",
    "    elements = statement.split(\" \")\n",
    "    elements_to_wrap = random.choices(list(range(len(elements))), k = 2)\n",
    "    for i in elements_to_wrap:\n",
    "        elements[i] = f\"<{elements[i]}>\"\n",
    "        \n",
    "    all_sentences.append(\" \".join(elements))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "181b0a40-1cdb-4f3f-a384-3e8c71848f1e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['In the last quarter, we <managed> to increase our revenue <by> 15% due to the successful launch of our new product line.',\n",
       " 'We anticipate that our investments <in> R&D will lead to a <20%> improvement in efficiency in the next two years.']"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "all_sentences[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4cbf629-652f-41d9-a160-b73f23b77f22",
   "metadata": {},
   "source": [
    "### Create completions\n",
    "\n",
    "Normally you would need to manually create a dataset (or obtain it some other way). \n",
    "\n",
    "But for this toy example we can programmatically generate the answer, which makes for an easier demo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "5527d7e9-8f0b-4fb8-bfab-cf6058b31424",
   "metadata": {},
   "outputs": [],
   "source": [
    "full_dataset = []\n",
    "for i, sen in enumerate(all_sentences):\n",
    "    wrapped_words = re.findall(\"<(.*?)>\", sen)\n",
    "    full_dataset.append({\n",
    "        \"i\" : i,\n",
    "        \"text\" : sen,\n",
    "        \"word_hits\" : wrapped_words\n",
    "    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "458d6949-9981-445a-93a7-a91c3891e71b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'i': 0,\n",
       "  'text': 'In the last quarter, we <managed> to increase our revenue <by> 15% due to the successful launch of our new product line.',\n",
       "  'word_hits': ['managed', 'by']},\n",
       " {'i': 1,\n",
       "  'text': 'We anticipate that our investments <in> R&D will lead to a <20%> improvement in efficiency in the next two years.',\n",
       "  'word_hits': ['in', '20%']}]"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "full_dataset[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "62fa5062-ab14-445e-aa8e-d47884bd5981",
   "metadata": {},
   "source": [
    "### Create prompt and completion\n",
    "\n",
    "We don't have to include instructions as part of the prompt as we are \"communicating\" the instructions through the examples that we show the model during fine-tuning."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "f3e28001-62ea-446d-a748-2a79adab4079",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_template = \"\"\"Text:\n",
    "{text}\n",
    "####\n",
    "\"\"\"\n",
    "\n",
    "completion_template = \"\"\"\n",
    "{word_hits}\n",
    "<|end|>\n",
    "\"\"\".strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "aedfc196-858f-48a3-80b8-07adea146a87",
   "metadata": {},
   "outputs": [],
   "source": [
    "for row in full_dataset:\n",
    "    row[\"prompt\"] = prompt_template.format(**row)\n",
    "\n",
    "    row[\"completion\"] = completion_template.format(**{\n",
    "         \"word_hits\" : json.dumps(row[\"word_hits\"])\n",
    "    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "08614044-6c84-41a3-a6cb-9b3147bb1215",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text:\n",
      "In the last quarter, we <managed> to increase our revenue <by> 15% due to the successful launch of our new product line.\n",
      "####\n",
      "[\"managed\", \"by\"]\n",
      "<|end|>\n"
     ]
    }
   ],
   "source": [
    "print(full_dataset[0][\"prompt\"]+full_dataset[0][\"completion\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67200c79-9fd8-4528-bfcf-9288b89b0b19",
   "metadata": {},
   "source": [
    "*Note:* we use `####` to indicate where the prompt ends and the completion starts. And we use `<|end|>` to indicate where the completion ends. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46338ad1-3d1f-47b7-840c-a36165202abb",
   "metadata": {},
   "source": [
    "---\n",
    "## Fine-tune\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52f81ec3-f2aa-43e6-b983-c79d93252ad8",
   "metadata": {},
   "source": [
    "### Create training and eval splits"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "8c47f492-ff8a-40f8-8cc8-2dfe2c6cee4e",
   "metadata": {},
   "outputs": [],
   "source": [
    "random.shuffle(full_dataset)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "1630d85c-a242-4d33-b224-876d0974d974",
   "metadata": {},
   "outputs": [],
   "source": [
    "training_dataset = full_dataset[:50]\n",
    "eval_dataset = full_dataset[50:]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a02d104-18be-4148-ba55-a74da74c318c",
   "metadata": {},
   "source": [
    "### Create training file\n",
    "\n",
    "The most common format for uploading training datasets is a `.jsonl` file. Which is a text file where every row is seperated by a newline and the content of every row is formatted as json."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "dd3990ff-a072-4923-8fc9-3798c68a7f69",
   "metadata": {},
   "outputs": [],
   "source": [
    "train_input = []\n",
    "for item in training_dataset:\n",
    "    train_input.append({\n",
    "            \"messages\" : [\n",
    "                {\"role\": \"user\", \"content\": item[\"prompt\"]},\n",
    "                {\"role\": \"assistant\", \"content\": item[\"completion\"]}\n",
    "            ]\n",
    "        })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "9671ff9f-e71f-4022-936b-8e5ff6bb533d",
   "metadata": {},
   "outputs": [],
   "source": [
    "ft_jsonl_file = Path.cwd() / \"data/ft_input.jsonl\" \n",
    "\n",
    "with open(ft_jsonl_file, \"w\", encoding = \"utf-8\") as f:\n",
    "    for item in train_input:\n",
    "        f.write(json.dumps(item) + \"\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b61313ba-d9dc-4df2-9ab2-afc087e6871a",
   "metadata": {},
   "source": [
    "### Validate training data\n",
    "\n",
    "The code below is provided by OpenAI through their documention."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "6e4dc40c-e4c0-4ac5-908d-9f59da77624e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Num examples: 50\n",
      "First example:\n",
      "{'role': 'user', 'content': 'Text:\\nWe project that our ongoing efforts to <expand> our international presence will result in a 20% increase in global revenue over the <next> three years.\\n####\\n'}\n",
      "{'role': 'assistant', 'content': '[\"expand\", \"next\"]\\n<|end|>'}\n",
      "No errors found\n",
      "Num examples missing system message: 50\n",
      "Num examples missing user message: 0\n",
      "\n",
      "#### Distribution of num_messages_per_example:\n",
      "min / max: 2, 2\n",
      "mean / median: 2.0, 2.0\n",
      "p5 / p95: 2.0, 2.0\n",
      "\n",
      "#### Distribution of num_total_tokens_per_example:\n",
      "min / max: 49, 67\n",
      "mean / median: 55.46, 55.0\n",
      "p5 / p95: 50.0, 59.2\n",
      "\n",
      "#### Distribution of num_assistant_tokens_per_example:\n",
      "min / max: 9, 13\n",
      "mean / median: 11.28, 11.0\n",
      "p5 / p95: 11.0, 12.0\n",
      "\n",
      "0 examples may be over the 4096 token limit, they will be truncated during fine-tuning\n",
      "Dataset has ~2773 tokens that will be charged for during training\n",
      "By default, you'll train for 4 epochs on this dataset\n",
      "By default, you'll be charged for ~11092 tokens\n",
      "See pricing page to estimate total costs\n"
     ]
    }
   ],
   "source": [
    "# We start by importing the required packages\n",
    "\n",
    "import json\n",
    "import os\n",
    "import tiktoken\n",
    "import numpy as np\n",
    "from collections import defaultdict\n",
    "\n",
    "# Next, we specify the data path and open the JSONL file\n",
    "\n",
    "data_path = (ft_jsonl_file).as_posix().replace(\"/\", \"\\\\\")\n",
    "\n",
    "# Load dataset\n",
    "with open(data_path) as f:\n",
    "    dataset = [json.loads(line) for line in f]\n",
    "\n",
    "# We can inspect the data quickly by checking the number of examples and the first item\n",
    "\n",
    "# Initial dataset stats\n",
    "print(\"Num examples:\", len(dataset))\n",
    "print(\"First example:\")\n",
    "for message in dataset[0][\"messages\"]:\n",
    "    print(message)\n",
    "\n",
    "# Now that we have a sense of the data, we need to go through all the different examples and check to make sure the formatting is correct and matches the Chat completions message structure\n",
    "\n",
    "# Format error checks\n",
    "format_errors = defaultdict(int)\n",
    "\n",
    "for ex in dataset:\n",
    "    if not isinstance(ex, dict):\n",
    "        format_errors[\"data_type\"] += 1\n",
    "        continue\n",
    "\n",
    "    messages = ex.get(\"messages\", None)\n",
    "    if not messages:\n",
    "        format_errors[\"missing_messages_list\"] += 1\n",
    "        continue\n",
    "\n",
    "    for message in messages:\n",
    "        if \"role\" not in message or \"content\" not in message:\n",
    "            format_errors[\"message_missing_key\"] += 1\n",
    "\n",
    "        if any(k not in (\"role\", \"content\", \"name\") for k in message):\n",
    "            format_errors[\"message_unrecognized_key\"] += 1\n",
    "\n",
    "        if message.get(\"role\", None) not in (\"system\", \"user\", \"assistant\"):\n",
    "            format_errors[\"unrecognized_role\"] += 1\n",
    "\n",
    "        content = message.get(\"content\", None)\n",
    "        if not content or not isinstance(content, str):\n",
    "            format_errors[\"missing_content\"] += 1\n",
    "\n",
    "    if not any(message.get(\"role\", None) == \"assistant\" for message in messages):\n",
    "        format_errors[\"example_missing_assistant_message\"] += 1\n",
    "\n",
    "if format_errors:\n",
    "    print(\"Found errors:\")\n",
    "    for k, v in format_errors.items():\n",
    "        print(f\"{k}: {v}\")\n",
    "else:\n",
    "    print(\"No errors found\")\n",
    "\n",
    "# Beyond the structure of the message, we also need to ensure that the length does not exceed the 4096 token limit.\n",
    "\n",
    "# Token counting functions\n",
    "encoding = tiktoken.get_encoding(\"cl100k_base\")\n",
    "\n",
    "# not exact!\n",
    "# simplified from https://github.com/openai/openai-cookbook/blob/main/examples/How_to_count_tokens_with_tiktoken.ipynb\n",
    "def num_tokens_from_messages(messages, tokens_per_message=3, tokens_per_name=1):\n",
    "    num_tokens = 0\n",
    "    for message in messages:\n",
    "        num_tokens += tokens_per_message\n",
    "        for key, value in message.items():\n",
    "            num_tokens += len(encoding.encode(value))\n",
    "            if key == \"name\":\n",
    "                num_tokens += tokens_per_name\n",
    "    num_tokens += 3\n",
    "    return num_tokens\n",
    "\n",
    "def num_assistant_tokens_from_messages(messages):\n",
    "    num_tokens = 0\n",
    "    for message in messages:\n",
    "        if message[\"role\"] == \"assistant\":\n",
    "            num_tokens += len(encoding.encode(message[\"content\"]))\n",
    "    return num_tokens\n",
    "\n",
    "def print_distribution(values, name):\n",
    "    print(f\"\\n#### Distribution of {name}:\")\n",
    "    print(f\"min / max: {min(values)}, {max(values)}\")\n",
    "    print(f\"mean / median: {np.mean(values)}, {np.median(values)}\")\n",
    "    print(f\"p5 / p95: {np.quantile(values, 0.1)}, {np.quantile(values, 0.9)}\")\n",
    "\n",
    "# Last, we can look at the results of the different formatting operations before proceeding with creating a fine-tuning job:\n",
    "\n",
    "# Warnings and tokens counts\n",
    "n_missing_system = 0\n",
    "n_missing_user = 0\n",
    "n_messages = []\n",
    "convo_lens = []\n",
    "assistant_message_lens = []\n",
    "\n",
    "for ex in dataset:\n",
    "    messages = ex[\"messages\"]\n",
    "    if not any(message[\"role\"] == \"system\" for message in messages):\n",
    "        n_missing_system += 1\n",
    "    if not any(message[\"role\"] == \"user\" for message in messages):\n",
    "        n_missing_user += 1\n",
    "    n_messages.append(len(messages))\n",
    "    convo_lens.append(num_tokens_from_messages(messages))\n",
    "    assistant_message_lens.append(num_assistant_tokens_from_messages(messages))\n",
    "\n",
    "print(\"Num examples missing system message:\", n_missing_system)\n",
    "print(\"Num examples missing user message:\", n_missing_user)\n",
    "print_distribution(n_messages, \"num_messages_per_example\")\n",
    "print_distribution(convo_lens, \"num_total_tokens_per_example\")\n",
    "print_distribution(assistant_message_lens, \"num_assistant_tokens_per_example\")\n",
    "n_too_long = sum(l > 4096 for l in convo_lens)\n",
    "print(f\"\\n{n_too_long} examples may be over the 4096 token limit, they will be truncated during fine-tuning\")\n",
    "\n",
    "# Pricing and default n_epochs estimate\n",
    "MAX_TOKENS_PER_EXAMPLE = 4096\n",
    "\n",
    "MIN_TARGET_EXAMPLES = 100\n",
    "MAX_TARGET_EXAMPLES = 25000\n",
    "TARGET_EPOCHS = 4\n",
    "MIN_EPOCHS = 1\n",
    "MAX_EPOCHS = 25\n",
    "\n",
    "n_epochs = TARGET_EPOCHS\n",
    "n_train_examples = len(dataset)\n",
    "if n_train_examples * TARGET_EPOCHS < MIN_TARGET_EXAMPLES:\n",
    "    n_epochs = min(MAX_EPOCHS, MIN_TARGET_EXAMPLES // n_train_examples)\n",
    "elif n_train_examples * TARGET_EPOCHS > MAX_TARGET_EXAMPLES:\n",
    "    n_epochs = max(MIN_EPOCHS, MAX_TARGET_EXAMPLES // n_train_examples)\n",
    "\n",
    "n_billing_tokens_in_dataset = sum(min(MAX_TOKENS_PER_EXAMPLE, length) for length in convo_lens)\n",
    "print(f\"Dataset has ~{n_billing_tokens_in_dataset} tokens that will be charged for during training\")\n",
    "print(f\"By default, you'll train for {n_epochs} epochs on this dataset\")\n",
    "print(f\"By default, you'll be charged for ~{n_epochs * n_billing_tokens_in_dataset} tokens\")\n",
    "print(\"See pricing page to estimate total costs\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fd601e8-c8b0-46cf-814b-6b0608c68115",
   "metadata": {},
   "source": [
    "--- \n",
    "## Perform fine-tuning\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e17e68f7-1969-47ed-ae42-bb8df1e34ba4",
   "metadata": {},
   "source": [
    "#### Upload file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "61090ef2-17bc-4997-aff7-bbd62876c593",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<File file id=file-0AcXBsPI2RX9uu6ZkmiKmbxG at 0x27bddb06e80> JSON: {\n",
       "  \"object\": \"file\",\n",
       "  \"id\": \"file-0AcXBsPI2RX9uu6ZkmiKmbxG\",\n",
       "  \"purpose\": \"fine-tune\",\n",
       "  \"filename\": \"file\",\n",
       "  \"bytes\": 13839,\n",
       "  \"created_at\": 1696026310,\n",
       "  \"status\": \"uploaded\",\n",
       "  \"status_details\": null\n",
       "}"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "CREATE_FT_FILE = True\n",
    "if CREATE_FT_FILE:\n",
    "    _ = openai.File.create(\n",
    "      file=open(ft_jsonl_file.as_posix().replace(\"/\", \"\\\\\"), \"rb\"),\n",
    "      purpose='fine-tune'\n",
    "    )\n",
    "_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "83dd3c6c-a203-47d3-8c99-3e8e59c20def",
   "metadata": {},
   "source": [
    "#### Start fine-tuning"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "b4ba7c34-6719-43fa-b75e-a2800c30d68a",
   "metadata": {},
   "outputs": [],
   "source": [
    "RUN_FT = True\n",
    "file_id = \"file-0AcXBsPI2RX9uu6ZkmiKmbxG\"\n",
    "if RUN_FT:\n",
    "    _ = (\n",
    "        openai\n",
    "        .FineTuningJob\n",
    "        .create(\n",
    "            training_file = file_id,\n",
    "            model = \"gpt-3.5-turbo\",\n",
    "            hyperparameters = {\n",
    "                \"n_epochs\" : 2 ## For complex tasks you can increase this \n",
    "            },\n",
    "            suffix = \"openai_ft_demo\"\n",
    "        )\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f745944a-3ee0-4a5d-9549-e8a40d971e24",
   "metadata": {},
   "source": [
    "#### Track progress"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "b72f68e7-11fb-4154-8fbf-986ffeab552c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<OpenAIObject list at 0x27bdd4cfd80> JSON: {\n",
       "  \"object\": \"list\",\n",
       "  \"data\": [\n",
       "    {\n",
       "      \"object\": \"fine_tuning.job\",\n",
       "      \"id\": \"ftjob-TbN6ZHg3qRVQUQJRPKCL5iC7\",\n",
       "      \"model\": \"gpt-3.5-turbo-0613\",\n",
       "      \"created_at\": 1696026402,\n",
       "      \"finished_at\": null,\n",
       "      \"fine_tuned_model\": null,\n",
       "      \"organization_id\": \"org-qyGkcV6pKgERYlKgngXYNEFy\",\n",
       "      \"result_files\": [],\n",
       "      \"status\": \"running\",\n",
       "      \"validation_file\": null,\n",
       "      \"training_file\": \"file-0AcXBsPI2RX9uu6ZkmiKmbxG\",\n",
       "      \"hyperparameters\": {\n",
       "        \"n_epochs\": 2\n",
       "      },\n",
       "      \"trained_tokens\": null,\n",
       "      \"error\": null\n",
       "    }\n",
       "  ],\n",
       "  \"has_more\": true\n",
       "}"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "openai.FineTuningJob.list(limit=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "a401840b-772f-4726-a329-a2b9cbf2bf05",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<FineTuningJob fine_tuning.job id=ftjob-TbN6ZHg3qRVQUQJRPKCL5iC7 at 0x27bde4186d0> JSON: {\n",
       "  \"object\": \"fine_tuning.job\",\n",
       "  \"id\": \"ftjob-TbN6ZHg3qRVQUQJRPKCL5iC7\",\n",
       "  \"model\": \"gpt-3.5-turbo-0613\",\n",
       "  \"created_at\": 1696026402,\n",
       "  \"finished_at\": null,\n",
       "  \"fine_tuned_model\": null,\n",
       "  \"organization_id\": \"org-qyGkcV6pKgERYlKgngXYNEFy\",\n",
       "  \"result_files\": [],\n",
       "  \"status\": \"running\",\n",
       "  \"validation_file\": null,\n",
       "  \"training_file\": \"file-0AcXBsPI2RX9uu6ZkmiKmbxG\",\n",
       "  \"hyperparameters\": {\n",
       "    \"n_epochs\": 2\n",
       "  },\n",
       "  \"trained_tokens\": null,\n",
       "  \"error\": null\n",
       "}"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ft_job_id = \"ftjob-TbN6ZHg3qRVQUQJRPKCL5iC7\"\n",
    "openai.FineTuningJob.retrieve(ft_job_id)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "id": "3b7d513f-70a3-46cf-81e5-401388fcf861",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<OpenAIObject fine_tuning.job.event id=ftevent-TEXZkdoEeR0sSrnKNnEO2Zh6 at 0x27bde41b7e0> JSON: {\n",
       "   \"object\": \"fine_tuning.job.event\",\n",
       "   \"id\": \"ftevent-TEXZkdoEeR0sSrnKNnEO2Zh6\",\n",
       "   \"created_at\": 1696026759,\n",
       "   \"level\": \"info\",\n",
       "   \"message\": \"The job has successfully completed\",\n",
       "   \"data\": {},\n",
       "   \"type\": \"message\"\n",
       " },\n",
       " <OpenAIObject fine_tuning.job.event id=ftevent-o8f25XMzoIXYqKnlZilKmLjK at 0x27bde41b060> JSON: {\n",
       "   \"object\": \"fine_tuning.job.event\",\n",
       "   \"id\": \"ftevent-o8f25XMzoIXYqKnlZilKmLjK\",\n",
       "   \"created_at\": 1696026757,\n",
       "   \"level\": \"info\",\n",
       "   \"message\": \"New fine-tuned model created: ft:gpt-3.5-turbo-0613:ties-and-coauthors:openai-ft-demo:84GKDrg2\",\n",
       "   \"data\": {},\n",
       "   \"type\": \"message\"\n",
       " },\n",
       " <OpenAIObject fine_tuning.job.event id=ftevent-jnHyndxWDfVy6hqHEysol9AC at 0x27bddefc360> JSON: {\n",
       "   \"object\": \"fine_tuning.job.event\",\n",
       "   \"id\": \"ftevent-jnHyndxWDfVy6hqHEysol9AC\",\n",
       "   \"created_at\": 1696026728,\n",
       "   \"level\": \"info\",\n",
       "   \"message\": \"Step 91/100: training loss=0.00\",\n",
       "   \"data\": {\n",
       "     \"step\": 91,\n",
       "     \"train_loss\": 8.803147579783399e-07,\n",
       "     \"train_mean_token_accuracy\": 1.0\n",
       "   },\n",
       "   \"type\": \"metrics\"\n",
       " },\n",
       " <OpenAIObject fine_tuning.job.event id=ftevent-3K0h7nq7Foln8CWE2PmtyvWO at 0x27bdea1b4c0> JSON: {\n",
       "   \"object\": \"fine_tuning.job.event\",\n",
       "   \"id\": \"ftevent-3K0h7nq7Foln8CWE2PmtyvWO\",\n",
       "   \"created_at\": 1696026707,\n",
       "   \"level\": \"info\",\n",
       "   \"message\": \"Step 81/100: training loss=0.00\",\n",
       "   \"data\": {\n",
       "     \"step\": 81,\n",
       "     \"train_loss\": 9.5367431640625e-07,\n",
       "     \"train_mean_token_accuracy\": 1.0\n",
       "   },\n",
       "   \"type\": \"metrics\"\n",
       " },\n",
       " <OpenAIObject fine_tuning.job.event id=ftevent-HncaB6eQNHCRYeQZwC3psqya at 0x27bdea1b600> JSON: {\n",
       "   \"object\": \"fine_tuning.job.event\",\n",
       "   \"id\": \"ftevent-HncaB6eQNHCRYeQZwC3psqya\",\n",
       "   \"created_at\": 1696026687,\n",
       "   \"level\": \"info\",\n",
       "   \"message\": \"Step 71/100: training loss=0.00\",\n",
       "   \"data\": {\n",
       "     \"step\": 71,\n",
       "     \"train_loss\": 1.1737530485333991e-06,\n",
       "     \"train_mean_token_accuracy\": 1.0\n",
       "   },\n",
       "   \"type\": \"metrics\"\n",
       " }]"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "openai.FineTuningJob.list_events(id = ft_job_id)[\"data\"][:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5fd07f70-5285-4dc3-bf70-31ccc4687506",
   "metadata": {},
   "source": [
    "---\n",
    "## Use fine-tuned model\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12256d3a-cb43-4be2-a352-85603e101ac6",
   "metadata": {},
   "source": [
    "#### Helper function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "4c4fc111-bb9a-4b10-9e7d-50ae593519b7",
   "metadata": {},
   "outputs": [],
   "source": [
    "def make_prediction(\n",
    "    prompt, \n",
    "    model,\n",
    "    temperature = 0,\n",
    "    end = \"<|end|>\"\n",
    "    ):\n",
    "    \n",
    "    completion = openai.ChatCompletion.create(\n",
    "        model = model,\n",
    "        temperature = temperature,\n",
    "        stop = [end], \n",
    "        messages = [\n",
    "                {\"role\": \"user\", \"content\": prompt}\n",
    "            ]\n",
    "    )\n",
    "\n",
    "    result = dict(completion[\"choices\"][0][\"message\"])\n",
    "    prediction = json.loads(result[\"content\"].strip()) \n",
    "        \n",
    "    return prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "659d2121-133b-49e0-ae3c-e8462eaf529c",
   "metadata": {},
   "source": [
    "#### Create prompts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "id": "5bdae12b-ba7b-4fac-9f15-b1c5cb6e333d",
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt_list = []\n",
    "for item in eval_dataset:\n",
    "    prompt_list.append(\n",
    "        prompt_template.format(**item)\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "41b38ac3-c47b-4ccd-9470-6a10ec1eabb3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text:\n",
      "As <we> continue to navigate through uncertain economic conditions, we remain committed <to> delivering value to our customers and shareholders.\n",
      "####\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(prompt_list[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1486c11d-0c46-4474-96cf-72dce525aed3",
   "metadata": {},
   "source": [
    "#### Generate predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "ae288683-d43c-42bf-b679-eb3a9daa9d66",
   "metadata": {},
   "outputs": [],
   "source": [
    "model_id = \"ft:gpt-3.5-turbo-0613:ties-and-coauthors:openai-ft-demo:84GKDrg2\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "f660492f-bb86-4527-bf85-6b14740ae3d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "res_list = []\n",
    "for prompt in prompt_list:\n",
    "\n",
    "    res = make_prediction(\n",
    "        prompt = prompt,\n",
    "        model = model_id\n",
    "    )\n",
    "    \n",
    "    res_list.append({\n",
    "        \"prompt\" : prompt,\n",
    "        \"prediction\" : res\n",
    "    })"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "id": "c90864d3-7d30-4716-b31e-7e99f66db781",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Text:\n",
      "As <we> continue to navigate through uncertain economic conditions, we remain committed <to> delivering value to our customers and shareholders.\n",
      "####\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Prediction: `['we', 'to']`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----\n",
      "\n",
      "Text:\n",
      "<The> launch of our <latest> software solution last quarter has helped us gain significant market share in the industry.\n",
      "####\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Prediction: `['The', 'latest']`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----\n",
      "\n",
      "Text:\n",
      "Over the past six months, we have <learned> valuable lessons from the challenges we faced and have <implemented> strategic changes to ensure future success.\n",
      "####\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Prediction: `['learned', 'implemented']`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----\n",
      "\n",
      "Text:\n",
      "Through the implementation of lean <manufacturing> principles, we were able to <reduce> lead times by 12% and improve overall product quality in the past year.\n",
      "####\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Prediction: `['manufacturing', 'reduce']`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----\n",
      "\n",
      "Text:\n",
      "Our recent partnership with a <leading> <technology> provider has significantly improved our product offerings and competitiveness.\n",
      "####\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/markdown": [
       "Prediction: `['leading', 'technology']`"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "----\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for item in res_list[:5]:\n",
    "    print(item[\"prompt\"])\n",
    "    mprint(f\"\"\"Prediction: `{item[\"prediction\"]}`\"\"\")\n",
    "    print(\"\\n----\\n\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}