getting_started.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d808a92c-d36d-4291-a4ad-b12bceb62fec",
   "metadata": {},
   "source": [
    "# Getting Started Azure OpenAI with LangChain in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6689d50a-d463-4ad1-8968-180f93bd2f8e",
   "metadata": {},
   "source": [
    "The purpose of this notebook is to provide a step-by-step guide on how to set up with Azure OpenAI and use the available models through the LangChain framework in Python. Additionally, it will demonstrate, through examples, how to implement models and utilize available features."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8202de6-7d14-44e6-b378-879d26f95ce3",
   "metadata": {},
   "source": [
    "## Obtaining Keys and Endpoints"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3b21c348-6a01-41d4-9df5-0d1d633367ef",
   "metadata": {},
   "source": [
    "Using the [**Azure sign up page**](https://azure.microsoft.com/en-us/pricing/purchase-options/azure-account) one can sign up and create an Azure OpenAI resource, giving access to all necessary credentials."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff61b81a-5c29-4da1-818f-fe3c2b1f9280",
   "metadata": {},
   "source": [
    "## Setup\n",
    "#### Installing and Importing Dependencies\n",
    "Microsoft [**recommends**](https://github.com/microsoft/vscode-jupyter/wiki/Installing-Python-packages-in-Jupyter-Notebooks) using:\n",
    "\n",
    "-  %pip for installing within Jupyter or IDE environments.\n",
    "-  %conda for installing within Conda environments.\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "448ce3be-b98f-4d46-8815-36fda94e9d30",
   "metadata": {},
   "source": [
    "#### Install the most current version of `langchain` and `langchain_openai`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23716db9-d8c8-4682-b662-0de8c185bd07",
   "metadata": {},
   "source": [
    "`%pip install -U langchain langchain_openai`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7f634643-841b-47fb-ba55-ff058e863528",
   "metadata": {},
   "source": [
    "#### Install Packages in a Virtual Environment (Optional)\n",
    "Set up a virtual environment by going to your project directory and executing the following command. This will create a new virtual environment in a folder named `.venv`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59fa41c8-3535-4d9b-9c30-59e3416ff875",
   "metadata": {},
   "source": [
    "**MacOS/UNIX**\n",
    "- `python3 -m venv .venv`\n",
    "\n",
    "**Windows**\n",
    "- `py -m venv .venv`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4679ec04-7fe4-49db-9887-2a821943f029",
   "metadata": {},
   "source": [
    "#### Activating the Virtual Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "74c7218d-e68c-4af6-8600-9bb523490392",
   "metadata": {},
   "source": [
    "To use the virtual environment, you must first activate it by executing the following command."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38cba946-94fd-445c-b23e-eea68661826d",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "**MacOS/UNIX**\n",
    "- `source .venv/bin/activate`\n",
    "\n",
    "  \n",
    "**Windows**\n",
    "- `.venv\\Scripts\\activate`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "994e5683-d8d4-4f37-945e-19b1d9aab967",
   "metadata": {},
   "source": [
    "#### Deactivating the Virtual Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5c481e4e-4c39-46ba-98c1-92448da66e93",
   "metadata": {},
   "source": [
    "If you want to leave the virtual environment in MacOS/UNIX and windows, simply execute the following command in the terminal:\n",
    "\n",
    "`deactivate`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46149db2-b280-45d5-b910-df1db63040bf",
   "metadata": {},
   "source": [
    "#### Set all required Environment Variables"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bf3454d",
   "metadata": {},
   "source": [
    "Create a `.env` file and store your credentials like follows:\n",
    "\n",
    "`AZURE_OPENAI_API_KEY` = <\"Your key here\">\n",
    "\n",
    "`AZURE_OPENAI_ENDPOINT` = <\"Your endpoint here\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7723ed6a",
   "metadata": {},
   "source": [
    "\n",
    "#### Or use dotenv to set and load all required env variables into/from your `.env` file.\n",
    "\n",
    "`%pip install python-dotenv`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "471ae21c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(True, 'COMPLETIONS_MODEL', 'Your model here')"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from dotenv import set_key\n",
    "set_key(\".env\", \"OPENAI_API_VERSION\", \"Your api version here\")\n",
    "set_key(\".env\", \"COMPLETIONS_MODEL\", \"Your model here\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e595a7e-9b33-472d-96ff-e3229d723642",
   "metadata": {},
   "source": [
    "#### Get all required Environment Variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "510fb595",
   "metadata": {},
   "outputs": [],
   "source": [
    "from dotenv import load_dotenv"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "fb74be9e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "load_dotenv()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b456baeb-fe51-45c8-940e-7f6ec52da0df",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Setting up the deployment name\n",
    "DEPLOYMENT_NAME = os.environ['COMPLETIONS_MODEL']\n",
    "\n",
    "# The API key for your Azure OpenAI resource.\n",
    "API_KEY = os.environ['AZURE_OPENAI_API_KEY']\n",
    "\n",
    "# The base URL for your Azure OpenAI resource. e.g. \"https://<your resource name>.openai.azure.com\"\n",
    "ENDPOINT = os.environ['AZURE_OPENAI_ENDPOINT']\n",
    "\n",
    "# The API version required\n",
    "VERSION = os.environ['OPENAI_API_VERSION']"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a43b7040-d6ab-4df8-8432-907eb1ca4099",
   "metadata": {},
   "source": [
    "## Creating an AzureChatOpenAI Model\n",
    "\n",
    "LangChain's [**AzureChatOpenAI Info**](https://python.langchain.com/v0.2/docs/integrations/chat/azure_chat_openai/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "035bd2cf-c045-4cdc-a0d1-f9bf60314f8d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import AzureChatOpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "172e4ff3-b3fd-4f10-bd2b-18ac1e259973",
   "metadata": {},
   "source": [
    "- Environment variables can be manually passed as parameters\n",
    "- The constuctor can search for environment variables of the corresponding names"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "74baa57a-6b35-403c-9902-2f5a9c7e2bd4",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = AzureChatOpenAI(\n",
    "    openai_api_version=VERSION,  \n",
    "    azure_deployment=DEPLOYMENT_NAME,\n",
    "    azure_endpoint=ENDPOINT,\n",
    "    temperature=0.5,\n",
    "    max_tokens=200,\n",
    "    timeout=60,\n",
    "    max_retries=10,\n",
    "    # organization=\"...\",\n",
    "    # model=\"gpt-35-turbo\",\n",
    "    # model_version=\"0125\",\n",
    "    # other params...\n",
    "    )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23b80bbe-442c-4374-a885-87bd6408dc28",
   "metadata": {},
   "source": [
    "**AZURE_OPENAI_API_VERSION** and **AZURE_OPENAI_API_ENDPOINT** are both being passed, but **AZURE_OPENAI_API_KEY** is being retrieved within the constructor."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b85766e-2581-4f24-86c1-cde10a0601c0",
   "metadata": {},
   "source": [
    "#### Other Optional Parameters\n",
    "\n",
    "- `temperature` determines how creative and unpredictable, or how deterministic and predictable, the model should be in its responses. A temperature of 0 would be predictable, while anything higher would make responses more random.\n",
    "\n",
    "- `max_tokens` defines the maximum number of tokens (words or pieces of words) the model can generate in its response.\n",
    "\n",
    "- `timeout` specifies the maximum amount of time (in seconds) to wait for a response from the API before timing out.\n",
    "\n",
    "- `max_retries` sets the number of times the API request should be retried in case of failure before giving up.\n",
    "\n",
    "- `organization` is used to specify the organization ID if required for API access.\n",
    "\n",
    "- `model` specifies the model to be used.\n",
    "\n",
    "- `model_version` indicates the specific version of the chosen model to use.\n",
    "\n",
    "- See the [**API Reference**](https://api.python.langchain.com/en/latest/chat_models/langchain_openai.chat_models.azure.AzureChatOpenAI.html) for more details\n",
    "\n",
    "- Other parameters may be available in different SDK's"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28a397b3-27b7-49f0-bccf-108f39196ee6",
   "metadata": {},
   "source": [
    "## Model Usage"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "7600d5eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.messages import HumanMessage, SystemMessage"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a152eab-52e5-48f3-bc8f-0b410d65c7d0",
   "metadata": {},
   "source": [
    "#### Using Messages from the `langchain_core.messages` Library\n",
    "\n",
    "The `langchain_core.messages` library allows the user to define messages for the model and assign roles to each message."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "259a462a-4a83-4f17-a337-be04d67d3020",
   "metadata": {},
   "source": [
    "- LangChain-compatible chat models take a list of `messages` as `input` and return the AI message as `output`.\n",
    "\n",
    "- All messages have `role` and `content` properties. However, below the roles are set using `SystemMessage` and `HumanMessage`. We'll cover more on this later.\n",
    "\n",
    "- Additional provider-specific information can be incorporated using the `Additional Properties`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "57c6f2ae-b6d0-46e3-831a-ad5f77daf0bd",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [\n",
    "    SystemMessage(content=\"Translate the following from German into English\"),\n",
    "    HumanMessage(content=\"Sie haben gerade Ihr erstes Kunstliche Itelligenz Model erstellt!\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "b6a40126-bede-46dc-8922-8721ac2f9c22",
   "metadata": {},
   "outputs": [],
   "source": [
    "response = model.invoke(messages)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "876d40ef-a75f-4d62-bc0a-43de6b55ce30",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'You have just created your first artificial intelligence model!'"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a0c37fe3-ed5d-4633-8e97-79cdba0e0365",
   "metadata": {},
   "source": [
    "### Prompting\n",
    "\n",
    "- Prompts are the inputs to language models, refined from raw user inputs to be ready for processing by the models.\n",
    "\n",
    "- [Prompting](https://www.datacamp.com/tutorial/prompt-engineering-with-langchain) involves crafting text inputs that clearly communicate with the models, outlining the specific task we want it to accomplish. This can include:\n",
    "    - Selecting the appropriate wording and setting a particular tone or style.\n",
    "    - Providing necessary context.\n",
    "    - Assigning a role, such as asking it to respond as if it were a native speaker of a certain language."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ee1ff1f-dbfc-445f-807e-4682db59460f",
   "metadata": {},
   "source": [
    "#### Prompt Templates\n",
    "\n",
    "- LangChain allows developers to design parameterized [**Prompt Templates**](https://python.langchain.com/v0.1/docs/modules/model_io/prompts/quick_start/) that are reusable and easily transferable between different models for integration.\n",
    "\n",
    "- It takes user input and inserts said input into the prompt to feed into the language models.\n",
    "\n",
    "#### `PromptTemplate`\n",
    "\n",
    "`PromptTemplate` is used to create an instance of [**Prompt**](https://python.langchain.com/v0.1/docs/expression_language/get_started/#1-prompt), and this is `invoked` by sending it to a model, which produces a `PromptValue`.\n",
    "\n",
    "The example code uses `.from_template`, which handles a single string template with placeholders for dynamic inputs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "04a11bb8-e4d9-4687-ab89-210aa1b7b51b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import PromptTemplate  \n",
    "\n",
    "prompt_template = PromptTemplate.from_template(\n",
    "    \"What vege crops can I grow in {month} in {city}, New Zealand?\"\n",
    ")\n",
    "\n",
    "prompt_value = prompt_template.format(month = \"December\", city = \"Rotorua\")\n",
    "\n",
    "# print(prompt_template) # <- uncomment to see\n",
    "# print(prompt_value) # <- uncomment to see"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f92db0c1-3741-42be-9d1e-edbe5010b61e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"In Rotorua, New Zealand, December falls in the Southern Hemisphere's summer, which is a great time for growing a variety of vegetables. Here are some vegetable crops you can plant in December:\\n\\n1. **Tomatoes**: Ideal for summer planting, they thrive in the warm weather.\\n2. **Capsicums (Bell Peppers)**: These also enjoy the summer heat.\\n3. **Zucchini**: Fast-growing and productive during warm months.\\n4. **Cucumbers**: Perfect for summer salads and pickling.\\n5. **Beans**: Both bush and pole beans grow well in the warm season.\\n6. **Sweet Corn**: Requires warm temperatures and plenty of sunlight.\\n7. **Pumpkins**: Plant now for a harvest in autumn.\\n8. **Eggplants**: Another heat-loving crop.\\n9. **Lettuce**: Opt for heat-tolerant varieties to avoid bolting.\\n10. **Radishes**: Fast-growing and can\""
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = model.invoke(prompt_value)\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ca904f4-faf1-4b95-bf77-7c7a7fa64515",
   "metadata": {},
   "source": [
    "#### `ChatPromptTemplate`\n",
    "\n",
    "This is optimized for a conversation-like format. The prompt is a list of chat messages. Each chat message is associated with `role` and `content`. In the example code, `.from_messages` is used to include multiple messages.\n",
    "\n",
    "Here, we will hardcode roles in the chat prompt, as opposed to using `SystemMessage` or `HumanMessage` like earlier."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "46e869bf-370d-4742-837f-c7d5eb76a891",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import ChatPromptTemplate\n",
    "\n",
    "chat_template = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        (\"system\", \"\"\"\n",
    "                    You are a travel agent helping customers plan their trips.\n",
    "                    Recommend them regarding popular Accommodation, Food and Activities of the country customer is asking.\n",
    "                    \"\"\"), \n",
    "        (\"ai\", \"Hi there, What can I help you with today?\"),\n",
    "        (\"human\", \"Hi I'm {name}, I'm planning a trip to {country}. Any recommendations\")\n",
    "    ]\n",
    ")\n",
    "\n",
    "prompt_value = chat_template.format_messages(name=\"Lucy\", country=\"New Zealand\")\n",
    "\n",
    "# print(chat_template) # <- uncomment to see\n",
    "# print(prompt_value) # <- uncomment to see"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c87adc53-1248-441d-aa46-44f410c24796",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\"Hi Lucy! New Zealand is an incredible destination with stunning landscapes, rich culture, and plenty of activities. Here are some recommendations for your trip:\\n\\n### Accommodation:\\n1. **Auckland:**\\n   - **SkyCity Hotel:** Located in the heart of the city, this hotel offers luxurious rooms and easy access to major attractions.\\n   - **Hotel DeBrett:** A boutique hotel with a unique design and a great location in the city center.\\n\\n2. **Queenstown:**\\n   - **Eichardt's Private Hotel:** Known for its elegance and stunning lake views.\\n   - **The Rees Hotel:** Offers luxurious accommodations with beautiful views of Lake Wakatipu.\\n\\n3. **Rotorua:**\\n   - **Regent of Rotorua:** A stylish boutique hotel with a central location.\\n   - **Solitaire Lodge:** A luxury lodge offering stunning views of Lake Tarawera.\\n\\n### Food:\\n1. **Auckland:**\\n   - **Depot Eatery & Oyster Bar:**\""
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = model.invoke(prompt_value)\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb6794d6-ba1f-4367-bae6-af0136212713",
   "metadata": {},
   "source": [
    "#### Assigning Roles Using LangChain Messages\n",
    "\n",
    "Compared to hardcoding the roles like above, LangChain Messages allow for more flexibility and better management, especially with complex conversations involving multiple roles. It also simplifies the visualization of the conversation flow.\n",
    "\n",
    "It is therefore recommended to use LangChain messages where possible.\n",
    "\n",
    "**Basic Message Types**\n",
    "\n",
    "|             |                                                                 |\n",
    "|-------------|-----------------------------------------------------------------|\n",
    "| `SystemMessage` | Set how the AI should behave (appropriate wording, tone, style, etc.) |\n",
    "| `HumanMessage`  | Message sent from the user                                      |\n",
    "| `AIMessage`     | Message from the AI chat model (context setting, guidance for response) |\n",
    "\n",
    "For more info, see [**Message Types**](https://python.langchain.com/v0.1/docs/modules/model_io/chat/message_types/) and [**API Reference**](https://api.python.langchain.com/en/latest/core_api_reference.html#module-langchain_core.messages)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95152680-ebc2-4106-99bf-679547d0d1fe",
   "metadata": {},
   "source": [
    "#### `base message` and `MessagePromptTemplate`\n",
    "We can also pass a `base message` or `MessagePromptTemplate` instead of tuples."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "b956d584-881a-4e9b-ae81-565ffdd7bf4a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import HumanMessagePromptTemplate\n",
    "\n",
    "chat_template = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        SystemMessage(\n",
    "            content=(\"You are a translator. You are to translate the text into English.\"\n",
    "            )\n",
    "        ),\n",
    "        HumanMessagePromptTemplate.from_template(\"{text}\"),\n",
    "    ]\n",
    ")\n",
    "\n",
    "prompt_value = chat_template.format_messages(text=\"アサヒスーパードライは日本のビールのです。\")\n",
    "\n",
    "# print(chat_template) # <- uncomment to see\n",
    "# print(prompt_value) # <- uncomment to see"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "648579a0-ca55-4885-bf70-47c0e08e066d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'Asahi Super Dry is a Japanese beer.'"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = model.invoke(prompt_value)\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "33f05d01-151b-4cda-a386-567a5523bb03",
   "metadata": {},
   "source": [
    "#### `MessagePlaceHolder`\n",
    "This  is used to select which messages to include when formatting."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "0f7c266b-4438-4912-9b21-b0611960f296",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.prompts import (\n",
    "    HumanMessagePromptTemplate,\n",
    "    SystemMessagePromptTemplate,\n",
    "    ChatPromptTemplate,\n",
    "    MessagesPlaceholder,\n",
    "    AIMessagePromptTemplate,\n",
    ")\n",
    "\n",
    "# SYSTEM ROLE Prompt\n",
    "system_template = SystemMessagePromptTemplate.from_template(\"\"\"\n",
    "                                            You are a precise assistant who knows the shcedule of the team.\n",
    "                                            Schedule details are as follows: {schedule}.\n",
    "                                            Only provide information to the team members.\n",
    "                                            Strictly only provide information specific to what is asked, Do not give extra information.\n",
    "                                            \"\"\")\n",
    "# HUMAN ROLE Prompt\n",
    "human_template = HumanMessagePromptTemplate.from_template(\"My name is {user_name}.\")\n",
    "# AI ROLE Prompt\n",
    "ai_template = AIMessagePromptTemplate.from_template(\"Hello {user_name}, how can I help you today?\")\n",
    "\n",
    "chat_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "        # this has essentially created a 'conversation history'\n",
    "        system_template,\n",
    "        human_template,\n",
    "        ai_template,\n",
    "        MessagesPlaceholder(variable_name=\"conversation\"), \n",
    "    ]\n",
    ")\n",
    "\n",
    "# print(chat_prompt) # <- uncomment to see the chat prompt "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ab8c223-97d6-4a5c-83a5-24eeba7c47f3",
   "metadata": {},
   "source": [
    "We can then input more prompts, which will take the `MessagePlaceholders`' place and create lines of sentences or a conversation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "cea2fce5-2692-4df1-bac1-80bb9472d7f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.messages import HumanMessage, AIMessage\n",
    "\n",
    "schedule = \"\"\"\n",
    "    Team Members: Alice, Bob, Carol, David, Emily\n",
    "    Team Meeting Schedule: Every Tuesday at 11:00 AM\n",
    "    Topic: LangChain with Azure OpenAI Integration\n",
    "\"\"\"\n",
    "# these messages will take MESSAGEPLACEHOLDERS place\n",
    "human_query = HumanMessage(\"When is the next team meeting and who is attending?\")\n",
    "ai_message = AIMessage(\"Hold on a second, let me check the schedule for you.\")\n",
    "\n",
    "prompt_value = chat_prompt.format_messages(\n",
    "    conversation=[human_query, ai_message], user_name=\"David\", schedule=schedule\n",
    ")\n",
    "\n",
    "# print(prompt_value) # <- uncomment to see the prompt "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "1558cfbf-5806-4a24-a0fa-8a1fb90a00d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'The next team meeting is on Tuesday at 11:00 AM. The attendees are Alice, Bob, Carol, David, and Emily.'"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "response = model.invoke(prompt_value)\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "34ec2ed1-83f1-4331-97fb-a6319f53e1fd",
   "metadata": {},
   "source": [
    "#### `FewShotPrompt`\n",
    "\n",
    "We can use examples (shots) to condition the model for a better response by including some example input and output in the prompt. This will inform the model about the context and how we want the output to be formatted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "fcaa0701-6f2c-4ffb-b011-75b6849d2ebe",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Currency Unit Conversion: [Input] one dollar => [Output] $1\n",
      "Currency Unit Conversion: [Input] one hundred yen => [Output] ¥100\n"
     ]
    }
   ],
   "source": [
    "from langchain_core.prompts import FewShotPromptTemplate\n",
    "\n",
    "examples = [\n",
    "    {\"input\": \"one dollar\", \"output\": \"$1\"},\n",
    "    {\"input\": \"thirty five euros\", \"output\": \"€35\"},\n",
    "]\n",
    "\n",
    "example_prompt = PromptTemplate(\n",
    "    input_variables=[\"input\", \"output\"], template=\"Currency Unit Conversion: [Input] {input} => [Output] {output}\"\n",
    ")\n",
    "\n",
    "# unpack the first example dictionary and feed it to the prompt template to format\n",
    "print(example_prompt.format(**examples[0]))\n",
    "\n",
    "# feed examples to FewShotPromptTemplate to generate a final prompt\n",
    "fewshot_prompt = FewShotPromptTemplate(\n",
    "    examples=examples,\n",
    "    example_prompt=example_prompt,\n",
    "    suffix= \"Convert the currency units: {input}\",\n",
    "    input_variables=[\"input\"],\n",
    ")\n",
    "\n",
    "prompt_value= fewshot_prompt.format(input=\"one hundred yen\")\n",
    "\n",
    "response = model.invoke(prompt_value)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e52b27f1-90a9-46c7-8a43-a8ae356235ef",
   "metadata": {},
   "source": [
    "## Chaining\n",
    "\n",
    "- Many LangChain components implement the [**Runnable**](https://python.langchain.com/v0.2/docs/concepts/#runnable-interface) protocol, which allows them to be easily chained together. These components can be combined in a sequence of calls, which we refer to as a chain.\n",
    "\n",
    "- Chaining `Runnables` in sequence."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "7bfdc351-1901-4fc4-bceb-067a68b54a30",
   "metadata": {},
   "outputs": [],
   "source": [
    "system_template = SystemMessagePromptTemplate.from_template(\"\"\"\n",
    "    You are an expert in {country} cuisine. \n",
    "    Keep it simple and short.\n",
    "    \"\"\")\n",
    "\n",
    "chat_prompt = ChatPromptTemplate.from_messages(\n",
    "    [\n",
    "         system_template,\n",
    "        (\"human\", \"I'd like to find out about {country} cuisine.\"),\n",
    "        (\"human\", \"{question}\"),\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f6682cc-576c-45f0-8b5b-52674a0f30a8",
   "metadata": {},
   "source": [
    "This is how we have been using prompts, but now we will skip this step and invoke using the chain."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "761b48c5-92e5-4857-b280-cdd486c29b02",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'In Japan, maguro (tuna) is the most popular sashimi. Globally, salmon sashimi tends to be more popular.'"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "prompt_value = chat_prompt.format_messages(country=\"Japanese\", question=\"What is the most popular Sashimi in Japan vs the rest of the world?\")\n",
    "\n",
    "response = model.invoke(prompt_value)\n",
    "response.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "3893edc8-acc0-418e-ac5d-31a65b00b8b8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "In Japan, the most popular sashimi is often Maguro (tuna), specifically the fatty part known as Otoro. Globally, Salmon sashimi tends to be more popular due to its rich flavor and widespread availability.\n"
     ]
    }
   ],
   "source": [
    "from langchain_core.output_parsers import StrOutputParser\n",
    "\n",
    "chain = chat_prompt | model | StrOutputParser()\n",
    "\n",
    "print(\n",
    "    chain.invoke(\n",
    "        {\n",
    "            \"country\": \"Japanese\", \n",
    "            \"question\": \"What is the most popular Sashimi in Japan vs the rest of the world?\"\n",
    "        }\n",
    "    )\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cceb7151",
   "metadata": {},
   "source": [
    "## Streaming Chat\n",
    "\n",
    "Azure OpenAI chat models also support a `Streaming Chat` feature. This feature allows for text to be received sentence by sentence, rather than waiting for the entire response to arrive."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "364f11f1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Certainly! The story of Papatuanuku and Ranginui is a central creation myth in Māori cosmology, explaining the origins of the world and the natural order. Here is a retelling of this profound and beautiful story:\n",
      "\n",
      "In the beginning, there was darkness, Te Pō, and within this darkness, two primal beings existed: Ranginui, the Sky Father, and Papatuanuku, the Earth Mother. They were locked in a tight, loving embrace, and between them, there was no space for light to enter. Their many children were born into this darkness, confined in the narrow space between their parents.\n",
      "\n",
      "The children of Ranginui and Papatuanuku were gods, each with their own domains and powers. They grew restless and yearned for light and space to live freely. The gods convened to discuss their plight and debated how they might gain freedom. Some suggested that their parents should be separated, while others were horrified at the thought of breaking their parents' embrace.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for chunk in model.stream('Tell me the story of Papatuanuku and Ranginui'):\n",
    "    print(chunk.content, end=\"\", flush=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f186179-13e7-4056-9b62-9b4b1ac8de60",
   "metadata": {},
   "source": [
    "### Wait in between API calls"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f747f98-7b78-436f-a889-e3cff3f19bf9",
   "metadata": {},
   "source": [
    "Depending on the users Azure plan and account will determine how many API requests can be made within a given interval. If too many requests are sent frequently an error may occur where the user will be told to wait **x** amount of time before sending another request.\n",
    "\n",
    "When creating a model, one of the parameters briefly mentioned earlier was the `max_retries` parameter. When calling the underlying Python OpenAI library, the library will wait and retry the call on your behalf at least 2 times (the default) before raising a RateLimiteError, or whatever `max retries` is set as.\n",
    "\n",
    "Otherwise, to ensure the user is not overwhelming the Azure services, make sure to wait between API calls when possible - this can be enforced by using the `time` package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "99e45cb7-6e35-4420-bd41-651157e5596f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import time\n",
    "time.sleep(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4a85fe6-ee18-4fc8-88c2-b67297ac8ef8",
   "metadata": {},
   "source": [
    "### Check the costs and token usage of a given model API call"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "id": "561dfbfb-0933-4650-8230-23f1bc2b1f46",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.callbacks.manager import get_openai_callback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "id": "c9081591-0b58-44b3-89c7-c3a512f84f4c",
   "metadata": {},
   "outputs": [],
   "source": [
    "messages = [\n",
    "    SystemMessage(content=\"Translate the following from German into English\"),\n",
    "    HumanMessage(content=\"What's the first planet in the Solar System?\"),\n",
    "]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61c9a128-f702-4f48-b695-436629e5ac1a",
   "metadata": {},
   "source": [
    "`get_openai_callback()` is a context manager of the [**OpenAICallbackHandler**](https://github.com/langchain-ai/langchain/blob/master/libs/langchain/langchain/callbacks/openai_info.py) class, meaning it calls this class and creates an instance when used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "178153e8-c37c-4ce2-bbb4-15ec6feb58f6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total tokens used: 37\n",
      "Total prompt tokens: 27\n",
      "Total prompt tokens: 10\n",
      "Total cost (in dollars): $0.000285\n",
      "Total successful requests): 1\n"
     ]
    }
   ],
   "source": [
    "with get_openai_callback() as cb:\n",
    "    model.invoke(messages)\n",
    "    print(f\"Total tokens used: {cb.total_tokens}\")\n",
    "    print(f\"Total prompt tokens: {cb.prompt_tokens}\")\n",
    "    print(f\"Total prompt tokens: {cb.completion_tokens}\")\n",
    "    print(f\"Total cost (in dollars): ${cb.total_cost}\")\n",
    "    print(f\"Total successful requests): {cb.successful_requests}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab9705c6-0512-48b3-b117-8e1bc385cf86",
   "metadata": {},
   "source": [
    "## Tools and Structured output"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e62770fa-c0e4-45ad-8f3b-5fd64e835857",
   "metadata": {},
   "source": [
    "Import the required packages. `BaseModel` is a parent class that all tools will inherit from, and `Field` is used to define all properties of the tool."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "84527cdd-6d45-4d18-a3c4-887a8537c9a5",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_core.pydantic_v1 import BaseModel, Field"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7b68b06-ccd1-40f1-9201-9814de191254",
   "metadata": {},
   "source": [
    "Tools are essentially classes that can be passed to a chosen model to influence or structure how the response should be formatted or generated.\n",
    "\n",
    "**For example:**\n",
    "\n",
    "- A Weather tool with a specific API call could be passed so the model knows to use this specific API for data retrieval.\n",
    "- A City tool with fields like `population`, `size`, and `main_language` so the model can return any city-related queries with an object containing the corresponding filled fields.\n",
    "- An Image tool with a `url` field to be returned when asked to search for an image containing a dog, with the field containing the URL of the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "05f0ff77-07bd-4041-9ab5-c2ba13ca644c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from typing import Optional\n",
    "#adds the ability to make a field Optional [NULL]\n",
    "\n",
    "class Person(BaseModel):\n",
    "    '''Information about a given person'''\n",
    "\n",
    "    name: str = Field(..., description=\"The name of a person\")\n",
    "    alive: bool = Field(..., description=\"Thether the person is alive or not\")\n",
    "    profession: str = Field(..., description=\"What the person does for work or professionally\")\n",
    "    noteable_features: str = Field(..., description=\"Any noteworthy features/achievements about the person\")\n",
    "    hobbies: Optional[str] = Field(description=\"Anything hobbies the person may have\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "id": "c5d9b7cc-d127-4f28-ab8f-829f236ef419",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Person(name='Kate Sheppard', alive=False, profession='Suffragist', noteable_features=\"Kate Sheppard was the most prominent member of the women's suffrage movement in New Zealand. She played a key role in making New Zealand the first country to grant women the right to vote in 1893.\", hobbies=None)"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "structured_model = model.with_structured_output(Person)\n",
    "response = structured_model.invoke(\"Tell me about Kate Sheppard\")\n",
    "response"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52eb76bf-283c-4095-aa33-27e772f09292",
   "metadata": {},
   "source": [
    "##### As the response of the invocation has been structured using the Person tool, the response can be accessed like a `Person` object."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "63f05dc8-7832-429c-b710-ceb72a56b492",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Kate Sheppard\n",
      "False\n",
      "Suffragist\n",
      "Kate Sheppard was the most prominent member of the women's suffrage movement in New Zealand. She played a key role in making New Zealand the first country to grant women the right to vote in 1893.\n"
     ]
    }
   ],
   "source": [
    "print(response.name)\n",
    "print(response.alive)\n",
    "print(response.profession)\n",
    "print(response.noteable_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4d5247e4-c810-4a61-98ee-9534ae10d300",
   "metadata": {},
   "source": [
    "#### JSON"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "2f688856-6992-4ed5-bf4b-62ddc592aee3",
   "metadata": {},
   "outputs": [],
   "source": [
    "import json "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "350506c9-8d59-43c5-a5ba-cc2493da2faf",
   "metadata": {},
   "source": [
    "Models can also be explicitly told to respond in a JSON structured format. This could then be used for future API calls or for easier access to information. However, **the word \"json\" must be included in the message string.**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "42478d15-0aa2-4e47-b279-e807397d9f31",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'{\\n  \"name\": \"Jane Doe\",\\n  \"alive\": true,\\n  \"profession\": \"Software Engineer\"\\n}'"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "json_model = model.bind(response_format={\"type\": \"json_object\"})\n",
    "\n",
    "response = json_model.invoke(\n",
    "    '''Return a JSON object of a random person with features like name,\n",
    "    alive (if they're alive or not) and their profession.''' \n",
    ")\n",
    "\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f656cd34",
   "metadata": {},
   "source": [
    "The response can then be formatted into a JSON object and accessed using normal JSON notation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0df62869",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'name': 'Jane Doe', 'alive': True, 'profession': 'Software Engineer'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "person = json.loads(response.content)\n",
    "person"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "535b4a4c-b435-40d1-9145-3918661fff7d",
   "metadata": {},
   "source": [
    "## File input"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8d150455-42c3-4943-89fa-d916ef76060b",
   "metadata": {},
   "source": [
    "Models can be fed files of various forms as their inputs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd54937c-eae4-4adc-be21-3779b6281e01",
   "metadata": {},
   "source": [
    "#### Images\n",
    "\n",
    "Import the required libraries to allow for HTTP requests and file conversion.\n",
    "\n",
    "- Make sure the file format is correct; otherwise, a \"File not found\" error will be thrown.\n",
    "\n",
    "- Ensure the HTTP request was successful with a 200 response code; otherwise, a 404 error will be thrown."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "ebd83bd8-da43-4c2b-a07f-a1bd0ecd0d19",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The status code is: 200\n"
     ]
    }
   ],
   "source": [
    "import base64\n",
    "import httpx\n",
    "\n",
    "url = \"https://upload.wikimedia.org/wikipedia/commons/b/bf/Aoraki_Mount_Cook.JPG\"\n",
    "\n",
    "# Use an HTTP get request to retrieve the image\n",
    "response = httpx.get(url)\n",
    "\n",
    "# Check the request was successful\n",
    "if response.status_code == 200:\n",
    "    # Extract the binary image data\n",
    "    binary_image_data = response.content\n",
    "\n",
    "    # Encode the binary_image_data into base 64 so that it's in bytes\n",
    "    # This allows it to be transmitted in JSON or text format\n",
    "    byte_image_data = base64.b64encode(binary_image_data)\n",
    "\n",
    "    # Decode the data into UTF-8 to allow for to make the data workable\n",
    "    # Web applications and models require it in a workable format\n",
    "    # UTF-8 is a fairly standardised format\n",
    "    image_data = byte_image_data.decode(\"utf-8\")\n",
    "\n",
    "print(f'The status code is: {response.status_code}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65f20898-73d6-4fed-9433-bb086019f1a7",
   "metadata": {},
   "source": [
    "When using data of different types, such as text in the `SystemMessage` and a file in the `HumanMessage`, it's necessary to specify a type header so the model knows how to interpret the data.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "fdb153be-bcb0-4a61-902a-3394cfb1206d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'This image depicts a stunning mountainous landscape with a snow-capped peak reflected in a calm, clear lake. The mountains are rugged and covered in snow, suggesting a cold and possibly remote location. The clear blue sky and the reflection in the water add to the serene and picturesque quality of the scene. The foreground features some rocks along the edge of the lake, enhancing the natural beauty of the setting. This type of scenery is often found in high-altitude regions, such as the Himalayas, the Alps, or the Southern Alps in New Zealand.'"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "messages = [  \n",
    "    SystemMessage(content=[{\"type\": \"text\", \"text\": \"describe the image location\"}]),\n",
    "    HumanMessage(\n",
    "    content=[\n",
    "        {\n",
    "            \"type\": \"image_url\",\n",
    "            \"image_url\": {\"url\": f\"data:image/jpeg;base64,{image_data}\"},\n",
    "        },\n",
    "    ]\n",
    "    )\n",
    "]\n",
    "response = model.invoke(messages)\n",
    "response.content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0e97f80-aa24-4ff8-af52-0f5b955a2090",
   "metadata": {},
   "source": [
    "#### Audio"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc44d1e7-7051-411b-8f9d-65f6548bb1a0",
   "metadata": {},
   "source": [
    "Chat completion models are somewhat limited in how they work with audio files. Audio files cannot be transmitted to the model directly but must instead be transcribed using the [**Whisper API**](https://platform.openai.com/docs/guides/speech-to-text). The transcribed text can then be used with the model.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c2fcba8-2d47-40ff-84c2-079250568a6e",
   "metadata": {},
   "source": [
    "The Whisper API can be accessed through `langchain_community` services.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "dc1f2f70-b45e-4bdb-bc62-d723a3c3e2e7",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_community.document_loaders.parsers import OpenAIWhisperParser"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "257b4d2a-5dd4-4487-824f-f19c45595f95",
   "metadata": {},
   "source": [
    "## Embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c5e24a75-b54e-4449-a788-4791047fb5d9",
   "metadata": {},
   "source": [
    "Embeddings, particularly `AzureOpenAIEmbeddings`, are a natural language processing technique that converts text into mathematical or vector representations. These representations capture the semantic meaning of words, phrases, or entire texts. This transformation enables Azure OpenAI search services to utilize numerical similarities between texts, returning the most relevant search results for a given query."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a2589e4-fc30-4216-9ddd-b932715cdc1c",
   "metadata": {},
   "source": [
    "#### Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "afbc35d6-563a-4e59-9a0b-ab2e4b604c5c",
   "metadata": {},
   "source": [
    "Embeddings models use a different Azure resource, this model name will be set below as follows."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "c248118a-9f5d-4720-b7e9-614a2eabe8b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "EMBEDDINGS_MODEL = os.environ['EMBEDDINGS_MODEL']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "65c4d92f-d05c-4447-a49d-d3eb6904bb8c",
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain_openai import AzureOpenAIEmbeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84777968-457c-4e7d-9386-ef79cf103b3a",
   "metadata": {},
   "source": [
    "#### Create a model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "8d296775-37c1-47f4-a318-cdeb5b4c777c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[0.029870394617319107, -0.004167019855231047, 0.008153270930051804, -0.011176464147865772, -0.015433868393301964]\n"
     ]
    }
   ],
   "source": [
    "E_model = AzureOpenAIEmbeddings(model = EMBEDDINGS_MODEL, azure_endpoint=ENDPOINT)\n",
    "\n",
    "# random generated text\n",
    "text = [\"The sun sets behind the mountains, casting a warm glow over the valley below.\",\n",
    "        \"Advances in artificial intelligence are transforming industries across the globe.\",\n",
    "        \"The recipe calls for two cups of flour, one teaspoon of baking powder, and a pinch of salt.\",\n",
    "        \"Exercise is essential for maintaining a healthy body and mind, promoting overall well-being.\",\n",
    "        \"Space exploration continues to reveal the mysteries of the universe, pushing the boundaries of human knowledge.\",\n",
    "        \"The novel’s protagonist faces a moral dilemma that challenges their deepest beliefs.\",\n",
    "        \"Sustainable practices in agriculture are crucial for preserving our environment for future generations.\"\n",
    "       ] \n",
    "\n",
    "embeddings = E_model.embed_documents(text) \n",
    "print(embeddings[0][:5])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "88a151d4-6d6c-4b27-bd87-ce26ea44b096",
   "metadata": {},
   "source": [
    "#### Searching"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a2b8abe-397b-43c9-accc-c589d62bbb61",
   "metadata": {},
   "source": [
    "To leverage the power of embeddings we will transform the text into an easily accessable data structure known as a `DataFrame` from the `pandas` library."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18bd3a4c-cdfe-4048-b2a8-9afc32de5285",
   "metadata": {},
   "source": [
    "`%pip install pandas`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "660012ad-7acc-45e5-b91f-5c7ef415b94f",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "id": "765025f8-15de-4eee-88d2-6fb6dfd31920",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>0</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The sun sets behind the mountains, casting a w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Advances in artificial intelligence are transf...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The recipe calls for two cups of flour, one te...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Exercise is essential for maintaining a health...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Space exploration continues to reveal the myst...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The novel’s protagonist faces a moral dilemma ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sustainable practices in agriculture are cruci...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                   0\n",
       "0  The sun sets behind the mountains, casting a w...\n",
       "1  Advances in artificial intelligence are transf...\n",
       "2  The recipe calls for two cups of flour, one te...\n",
       "3  Exercise is essential for maintaining a health...\n",
       "4  Space exploration continues to reveal the myst...\n",
       "5  The novel’s protagonist faces a moral dilemma ...\n",
       "6  Sustainable practices in agriculture are cruci..."
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_text = pd.DataFrame(text)\n",
    "df_text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48b3ad3e-f17c-448a-9a23-d67bc0762709",
   "metadata": {},
   "source": [
    "Rename the columns and index to make the `DataFrame` clearer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "id": "711f849b-6012-4439-b33a-8b5d0f7d6c41",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_text.rename(columns={0:'text'}, inplace=True)\n",
    "df_text.index.name = \"index\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "id": "f4113077-f4b2-47ea-96b8-9a9bd2ec17b2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The sun sets behind the mountains, casting a w...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Advances in artificial intelligence are transf...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The recipe calls for two cups of flour, one te...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Exercise is essential for maintaining a health...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Space exploration continues to reveal the myst...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The novel’s protagonist faces a moral dilemma ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sustainable practices in agriculture are cruci...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    text\n",
       "index                                                   \n",
       "0      The sun sets behind the mountains, casting a w...\n",
       "1      Advances in artificial intelligence are transf...\n",
       "2      The recipe calls for two cups of flour, one te...\n",
       "3      Exercise is essential for maintaining a health...\n",
       "4      Space exploration continues to reveal the myst...\n",
       "5      The novel’s protagonist faces a moral dilemma ...\n",
       "6      Sustainable practices in agriculture are cruci..."
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "id": "7073a830-2f7d-4af9-9e68-f029ecde4aa8",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_text['Embeddings'] = df_text['text'].apply(E_model.embed_query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "id": "16da7bc1-9a72-4b5e-a309-c1e07af3b82e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>Embeddings</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The sun sets behind the mountains, casting a w...</td>\n",
       "      <td>[0.029870394617319107, -0.004167019855231047, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Advances in artificial intelligence are transf...</td>\n",
       "      <td>[-0.012673170305788517, -0.020036686211824417,...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>The recipe calls for two cups of flour, one te...</td>\n",
       "      <td>[0.012168226763606071, 0.018271654844284058, 0...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Exercise is essential for maintaining a health...</td>\n",
       "      <td>[-0.01102688442915678, -0.007886004634201527, ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Space exploration continues to reveal the myst...</td>\n",
       "      <td>[0.028899280354380608, -0.01474663894623518, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>The novel’s protagonist faces a moral dilemma ...</td>\n",
       "      <td>[0.013899882324039936, -0.02602808177471161, -...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>Sustainable practices in agriculture are cruci...</td>\n",
       "      <td>[-0.006976415403187275, -0.018390081822872162,...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    text  \\\n",
       "index                                                      \n",
       "0      The sun sets behind the mountains, casting a w...   \n",
       "1      Advances in artificial intelligence are transf...   \n",
       "2      The recipe calls for two cups of flour, one te...   \n",
       "3      Exercise is essential for maintaining a health...   \n",
       "4      Space exploration continues to reveal the myst...   \n",
       "5      The novel’s protagonist faces a moral dilemma ...   \n",
       "6      Sustainable practices in agriculture are cruci...   \n",
       "\n",
       "                                              Embeddings  \n",
       "index                                                     \n",
       "0      [0.029870394617319107, -0.004167019855231047, ...  \n",
       "1      [-0.012673170305788517, -0.020036686211824417,...  \n",
       "2      [0.012168226763606071, 0.018271654844284058, 0...  \n",
       "3      [-0.01102688442915678, -0.007886004634201527, ...  \n",
       "4      [0.028899280354380608, -0.01474663894623518, -...  \n",
       "5      [0.013899882324039936, -0.02602808177471161, -...  \n",
       "6      [-0.006976415403187275, -0.018390081822872162,...  "
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_text"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "501b5125-e306-44a2-b1ab-d8ad4e4b8f51",
   "metadata": {},
   "source": [
    "Here you can see that for each `text` it's corresponding embeddings have been set in the `Embeddings` column in the same row."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "3351dd97-c5d8-4a5c-b2c8-3b3c6a884538",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[-0.012673170305788517, -0.020036686211824417]"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "E_model.embed_query(text[1])[:2]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "702e72bb-be6b-477a-9b73-80a419b759ac",
   "metadata": {},
   "source": [
    "Now that we have the numerical representation, various Azure services can use this for searching. However, to demonstrate what happens 'under the hood,' we will implement a basic vector search manually."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2bab3fa-bb18-4228-82fc-6459e8b4ca0f",
   "metadata": {},
   "source": [
    "#### Cosine similarity"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aac27f37-e119-4b9a-bf29-be6f68129328",
   "metadata": {},
   "source": [
    "Cosine similarity measures the similarity between two vectors by evaluating the cosine of the angle between them. In essence, it determines how close two vector points or lines are to each other. Vectors that are closer in space typically share a closer semantic meaning according to the model. This principle forms the core functionality of vector-based search using embeddings."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bc883247-6423-477d-ad96-18438f4b2156",
   "metadata": {},
   "source": [
    "#### Setup\n",
    "\n",
    "The `numpy` library introduces some conveniant mathematical functions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22dc169b-2b7c-4f98-bec6-2fafaacb35a3",
   "metadata": {},
   "source": [
    "`%pip install numpy`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "e4eeb79f-ad1f-4039-a5e1-cf5f8eaf17b0",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from numpy.linalg import norm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "6655272c-87fa-47d8-b2d3-d97c09d1fec8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def cosine_similarity(text, query):\n",
    "    return np.dot(text, query) / (np.linalg.norm(text) * np.linalg.norm(query))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30a2778c-9068-416e-835d-78935f7b7c9f",
   "metadata": {},
   "source": [
    "#### Search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a325870f-a0fc-4a38-807e-bab86d44d87d",
   "metadata": {},
   "source": [
    "use embeddings to get the vector representation of a query."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "id": "03325c0a-18a7-40d6-8eed-73ca94094fcb",
   "metadata": {},
   "outputs": [],
   "source": [
    "query = 'What text talks about space or stars?'\n",
    "query_embedding = E_model.embed_query(query)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "2c2ae0a5-c5a2-4587-b414-389a65a3c74e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_text['Similarity'] = df_text['Embeddings'].apply(lambda text_embedding : cosine_similarity(text_embedding, query_embedding)) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "f7978cb2-a878-4acd-8ae7-e6abfbf56e16",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>text</th>\n",
       "      <th>Embeddings</th>\n",
       "      <th>Similarity</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>index</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Space exploration continues to reveal the myst...</td>\n",
       "      <td>[0.028899280354380608, -0.01474663894623518, -...</td>\n",
       "      <td>0.825995</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>The sun sets behind the mountains, casting a w...</td>\n",
       "      <td>[0.029870394617319107, -0.004167019855231047, ...</td>\n",
       "      <td>0.759168</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                    text  \\\n",
       "index                                                      \n",
       "4      Space exploration continues to reveal the myst...   \n",
       "0      The sun sets behind the mountains, casting a w...   \n",
       "\n",
       "                                              Embeddings  Similarity  \n",
       "index                                                                 \n",
       "4      [0.028899280354380608, -0.01474663894623518, -...    0.825995  \n",
       "0      [0.029870394617319107, -0.004167019855231047, ...    0.759168  "
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_text.sort_values(by='Similarity', ascending=False).head(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a84d962-f036-442b-a781-b07a872cab42",
   "metadata": {},
   "source": [
    "Et voilà! The text with the highest cosine similarity discusses space and space exploration, which closely aligns with our query. We also observe that the second-most similar text mentions the Sun. While it’s not an exact match to our query, the Sun is a star, making it contextually similar and resulting in a relatively high cosine similarity."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}