03-Quering-AOpenAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "d59d527f-1100-45ff-b051-5f7c9029d94d",
   "metadata": {},
   "source": [
    "# Queries with and without Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eb9a9444-dc90-4fc3-aea7-8ee918301aba",
   "metadata": {},
   "source": [
    "So far, you have your Search Engine loaded **from two different data sources in two diferent text-based indexes**, on this notebook we are going to try some example queries and then use Azure OpenAI service to see if we can get even better results.\n",
    "\n",
    "The idea is that a user can ask a question about Computer Science (first datasource/index) or about Covid (second datasource/index), and the engine will respond accordingly.\n",
    "This **Multi-Index** demo, mimics the scenario where a company loads multiple type of documents of different types and about completly different topics and the search engine must respond with the most relevant results."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f6c7e3-9037-4b1e-ae17-1deaa27b9c08",
   "metadata": {},
   "source": [
    "## Set up variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8e50b404-a061-49e7-a3c7-c6eabc98ff0f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "import urllib\n",
    "import requests\n",
    "import random\n",
    "import json\n",
    "from collections import OrderedDict\n",
    "from IPython.display import display, HTML, Markdown\n",
    "from langchain.chains import LLMChain\n",
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.llms import AzureOpenAI\n",
    "from langchain.chat_models import AzureChatOpenAI\n",
    "from langchain.vectorstores import FAISS\n",
    "from langchain.docstore.document import Document\n",
    "from langchain.chains.question_answering import load_qa_chain\n",
    "from langchain.chains.qa_with_sources import load_qa_with_sources_chain\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "\n",
    "from common.prompts import COMBINE_QUESTION_PROMPT, COMBINE_PROMPT, COMBINE_PROMPT_TEMPLATE\n",
    "from common.utils import (\n",
    "    get_search_results,\n",
    "    model_tokens_limit,\n",
    "    num_tokens_from_docs,\n",
    "    num_tokens_from_string\n",
    ")\n",
    "\n",
    "from dotenv import load_dotenv\n",
    "load_dotenv(\"credentials.env\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "2f2c22f8-79ab-405c-95e8-77a1978e53bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Setup the Payloads header\n",
    "headers = {'Content-Type': 'application/json','api-key': os.environ['AZURE_SEARCH_KEY']}\n",
    "params = {'api-version': os.environ['AZURE_SEARCH_API_VERSION']}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9297d29b-1f61-4dce-858e-bf4272172dba",
   "metadata": {},
   "source": [
    "## Multi-Index Search queries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "5a46e2d3-298a-4708-83de-9e108b1a117a",
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Text-based Indexes that we are going to query (from Notebook 01 and 02)\n",
    "index1_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index1_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c62ebb2-d7be-4bfb-b1ba-4db86c11839a",
   "metadata": {},
   "source": [
    "Try questions that you think might be answered or addressed in computer science papers in 2020-2021 or that can be addressed by medical publications about COVID in 2020-2021. Try comparing the results with the open version of ChatGPT.<br>\n",
    "The idea is that the answers using Azure OpenAI only looks at the information contained on these publications.\n",
    "\n",
    "**Example Questions you can ask**:\n",
    "- What is CLP?\n",
    "- How Markov chains work?\n",
    "- What are some examples of reinforcement learning?\n",
    "- What are the main risk factors for Covid-19?\n",
    "- What medicine reduces inflamation in the lungs?\n",
    "- Why Covid doesn't affect kids that much compared to adults?\n",
    "- Does chloroquine really works against covid?\n",
    "- Who won the 1994 soccer world cup? # This question should yield no answer if the system is correctly grounded"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "b9b53c14-19bd-451f-aa43-7ad27ccfeead",
   "metadata": {},
   "outputs": [],
   "source": [
    "QUESTION = \"Explain the difference between exploration and explotation\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6d925eb-7f9c-429e-a62a-4c37d7702caf",
   "metadata": {},
   "source": [
    "### Search on both indexes individually and aggragate results\n",
    "\n",
    "#### **Note**: \n",
    "In order to standarize the indexes, **there must be 8 mandatory fields present on each text-based index**: `id, title, content, chunks, language, name, location, vectorized`. This is so that each document can be treated the same along the code. Also, **all indexes must have a semantic configuration**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "faf2e30f-e71f-4533-ab52-27d048b80a89",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "Index: cogsrch-index-files Results Found: 470, Results Returned: 10\n",
      "200\n",
      "Index: cogsrch-index-csv Results Found: 42870, Results Returned: 10\n"
     ]
    }
   ],
   "source": [
    "agg_search_results = dict()\n",
    "\n",
    "for index in indexes:\n",
    "    search_payload = {\n",
    "        \"search\": QUESTION,\n",
    "        \"select\": \"id, title, chunks, language, name, location\",\n",
    "        \"queryType\": \"semantic\",\n",
    "        \"semanticConfiguration\": \"my-semantic-config\",\n",
    "        \"count\": \"true\",\n",
    "        \"speller\": \"lexicon\",\n",
    "        \"queryLanguage\": \"en-us\",\n",
    "        \"captions\": \"extractive\",\n",
    "        \"answers\": \"extractive\",\n",
    "        \"top\": \"10\"\n",
    "    }\n",
    "\n",
    "    r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + index + \"/docs/search\",\n",
    "                     data=json.dumps(search_payload), headers=headers, params=params)\n",
    "    print(r.status_code)\n",
    "\n",
    "    search_results = r.json()\n",
    "    agg_search_results[index]=search_results\n",
    "    print(\"Index:\", index, \"Results Found: {}, Results Returned: {}\".format(search_results['@odata.count'], len(search_results['value'])))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7fd0fe5-4ee0-42e2-a920-72b93a407389",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Display the top results (from both searches) based on the score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "9e938337-602d-4b61-8141-b8c92a5d91da",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<h4>Top Answers</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "\n",
      "\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<h4>Top Results</h4>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206262/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Balancing Exploration and Exploitation in Self-imitation Learning</a> - score: 2.96</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In the exploring stage, with the aid of intrinsic reward, the agent tends to explore unseen states and occasionally collect high rewarding experiences, while in the self-imitating stage, the agent learns to consistently reproduce such experiences and thus provides a better starting point for subsequent stages."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7135900/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Revelation through exploitation: the viral model for intracellular traffic</a> - score: 2.78</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Revelation through exploitation: the viral model for intracellular traffic. Enveloped viruses exploit the existing routes of membrane traffic to enter and leave the host cell. The similarity between viral envelopes and cellular membranes has allowed the use of animal viruses as probes to examine aspects of intracellular traffic.."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1016/j.ejmech.2016.11.027; https://www.ncbi.nlm.nih.gov/pubmed/27914364/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Structure-based exploration and exploitation of the S4 subsite of norovirus 3CL protease in the design of potent and permeable inhibitors.</a> - score: 2.22</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Structure-based exploration and exploitation of the S4 subsite of norovirus 3CL protease in the design of potent and permeable inhibitors.. Human noroviruses are the primary cause of epidemic and sporadic acute gastroenteritis."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3594938/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Dangerous liaisons: molecular basis for a syndemic relationship between Kaposi’s sarcoma and P. falciparum malaria</a> - score: 1.8</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Against this background, ongoing studies are rapidly constructing a fascinating new paradigm in which the major host receptors that control parasite invasion (Basigin/CD147) and cyto-adherence (CD36) are, surprisingly, also important targets for exploitation by KSHV."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/0006/0006012v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">arXiv:cs/0006012v1  [cs.CL]  5 Jun 2000</a> - score: 1.76</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "At this point it seems plausible that every dedu\r",
       "tionthat has been made \r",
       "an be attributed to some individual.1    We have just presented two motivating reasons for produ\r",
       "ing systems that are ableto understand human languages, in the guise of a single reason."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160249/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">GENSTYLE: exploration and analysis of DNA sequences with genomic signature</a> - score: 1.65</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "tools for the exploration and analysis of signatures allow (i) identification of the origin of dna segments (detection of rare species or species for which technical problems prevent fast characterization, such as micro-organisms with slow growth), (ii) analysis of the homogeneity of a genome and isolation of areas with novel functionality …"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1016/j.ciresp.2013.02.010; https://www.ncbi.nlm.nih.gov/pubmed/24559592/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Laparoscopic common bile duct exploration. Lessons learned after 200 cases.</a> - score: 1.65</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "INTRODUCTION Laparoscopic common bile duct exploration (LCBDE) is a reliable, reproducible and cost-effective treatment for common bile duct stones. Several techniques have been described for choledochotomy closure. AIMS To present our experience and the lessons learned in more than 200 cases of LCBDE."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1016/j.ijsu.2016.10.012; https://www.ncbi.nlm.nih.gov/pubmed/27743897/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Is contralateral exploration justified in endoscopic total extraperitoneal repair of clinical unilateral groin hernias - A Prospective cohort study.</a> - score: 1.63</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In contrast to the high incidence (6/46, 13%) of CMIH in the non-exploration cohort, there was only one metachronous occurrence (1/68, 1.4%) after negative contralateral exploration at a median follow-up of longer than 3 yrs (p = 0.02)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/0012/0012003v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">0012003v1.pdf</a> - score: 1.57</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "an essential difference between descartes and a straightforward reading of quantum mechanics is that in quantum mechanics the observing, thinking individual and the physical world are both fundamental poles that exist in relation to one another.1  quantum mechanics does not assume an independently existing physical world beyond what it allows an …"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1016/j.surg.2014.06.020; https://www.ncbi.nlm.nih.gov/pubmed/25239339/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">A simulator-based resident curriculum for laparoscopic common bile duct exploration.</a> - score: 1.54</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "BACKGROUND Laparoscopic common bile duct exploration (LCBDE) remains an underused treatment for choledocholithiasis, likely in part because of a lack of exposure to the procedure during surgery residency. In this study, we implemented a resident LCBDE curriculum using a previously validated procedural simulator."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.4293/jsls.2014.00184; https://www.ncbi.nlm.nih.gov/pubmed/25516702/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Transcystic approach to laparoscopic common bile duct exploration.</a> - score: 1.51</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "The choledochoscope was inserted into the gallbladder through the small opening in the fundus of the gallbladder extracorporeally and was advanced toward the common bile duct via the cystic duct under the guidance of both laparoscopic imaging and endoscopic imaging."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/0007/0007001v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Microsoft Word - constraint1.doc</a> - score: 1.48</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "Each of  those entities is programmed in a different module:   1. model, sets up the structure of the model, that is, it gives the environment of the  simulation: range of parameters, initialisations, alternative choices and basic  (backward chaining) rules for calculations. 2. prover, generates the dynamics of the simulation."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://doi.org/10.1007/s00464-009-0809-4; https://www.ncbi.nlm.nih.gov/pubmed/20044767/?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">Laparoscopic transcystic bile duct exploration: the treatment of first choice for common bile duct stones.</a> - score: 1.44</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "RESULTS There was no difference in age (49 +/- 15 vs. 57 +/- 19, p = 0.7), sex (79% vs. 82% females, p = 0.6), and ASA grade (1.9 +/- 1 vs. 1.8 +/- 1, p = 0.7)."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/0003/0003057v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">arXiv:cs/0003057v1  [cs.LO]  13 Mar 2000</a> - score: 1.33</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "It is aimed at allowing the exploration of knowledge bases. It is well-suited for debugging large knowledge bases, by exploiting the modularity of such designs. The package uses XSB as a pre-processing phase, where the well-founded semantics of a program, with respect to a given query, is computed."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/0004/0004001v1.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">arXiv:cs/0004001v1  [cs.AI]  3 Apr 2000</a> - score: 1.24</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "In the following, a solution is always meant in this formal sense.    1 INTRODUCTION 4  the environment and updates its internal state. Then the next cycle follows. It operates according to some function p. We split the input xk into a regular part x′k and a credit ck, often called reinforcement feedback."
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<h5><a href=\"https://demodatasetsp.blob.core.windows.net/arxivcs/0004/0004014v2.pdf?sv=2022-11-02&ss=bf&srt=sco&sp=rltfx&se=2024-10-02T01:02:07Z&st=2023-08-03T17:02:07Z&spr=https&sig=gLxStXFSY6X29OPpPDpBEhoQDdtJNDrMVExNYJ%2BhmBQ%3D\">0004014v2.pdf</a> - score: 1.03</h5>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "energy including controlled fusion, medical science and molecular biology including genetic engineering, space exploitation and cosmology, climate and earth science, materials including semiconductors and composites, machine intelligence and robotics, financial modeling and commerce, and information communication and human-machine interface will …"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(HTML('<h4>Top Answers</h4>'))\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['@search.answers']:\n",
    "        if result['score'] > 0.5: # Show answers that are at least 50% of the max possible score=1\n",
    "            display(HTML('<h5>' + 'Answer - score: ' + str(round(result['score'],2)) + '</h5>'))\n",
    "            display(HTML(result['text']))\n",
    "            \n",
    "print(\"\\n\\n\")\n",
    "display(HTML('<h4>Top Results</h4>'))\n",
    "\n",
    "content = dict()\n",
    "ordered_content = OrderedDict()\n",
    "\n",
    "\n",
    "for index,search_results in agg_search_results.items():\n",
    "    for result in search_results['value']:\n",
    "        if result['@search.rerankerScore'] > 1:# Show answers that are at least 25% of the max possible score=4\n",
    "            content[result['id']]={\n",
    "                                    \"title\": result['title'],\n",
    "                                    \"chunks\": result['chunks'],\n",
    "                                    \"language\": result['language'], \n",
    "                                    \"name\": result['name'], \n",
    "                                    \"location\": result['location'] ,\n",
    "                                    \"caption\": result['@search.captions'][0]['text'],\n",
    "                                    \"score\": result['@search.rerankerScore'],\n",
    "                                    \"index\": index\n",
    "                                    }\n",
    "    \n",
    "#After results have been filtered we will Sort and add them as an Ordered list\\n\",\n",
    "for id in sorted(content, key= lambda x: content[x][\"score\"], reverse=True):\n",
    "    ordered_content[id] = content[id]\n",
    "    url = str(ordered_content[id]['location']) + os.environ['BLOB_SAS_TOKEN']\n",
    "    title = str(ordered_content[id]['title']) if (ordered_content[id]['title']) else ordered_content[id]['name']\n",
    "    score = str(round(ordered_content[id]['score'],2))\n",
    "    display(HTML('<h5><a href=\"'+ url + '\">' + title + '</a> - score: '+ score + '</h5>'))\n",
    "    display(HTML(ordered_content[id]['caption']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "52a6d3e6-afb2-4fa7-96d3-69bc2373ded5",
   "metadata": {},
   "source": [
    "## Comments on Query results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84e02227-6a92-4944-86f8-6c1e38d90fe4",
   "metadata": {},
   "source": [
    "As seen above the semantic search feature of Azure Cognitive Search service is good. It gives us some answers and also the top results with the corresponding file and the paragraph where the answers is possible located.\n",
    "\n",
    "Let's see if we can make this better with Azure OpenAI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8df3e6d4-9a09-4b0f-b328-238738ccfaec",
   "metadata": {},
   "source": [
    "# Using Azure OpenAI\n",
    "\n",
    "To use OpenAI to get a better answer to our question, the thought process is simple: let's **give the answer and the content of the documents from the search result to the GPT model as context and let it provide a better response**.\n",
    "\n",
    "Now, before we do this, we need to understand a few things first:\n",
    "\n",
    "1) Chainning and Prompt Engineering\n",
    "2) Embeddings\n",
    "\n",
    "We will use a library call **LangChain** that wraps a lot of boiler plate code.\n",
    "Langchain is one library that does a lot of the prompt engineering for us under the hood, for more information see [here](https://python.langchain.com/en/latest/index.html)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "eea62a7d-7e0e-4a93-a89c-20c96560c665",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set the ENV variables that Langchain needs to connect to Azure OpenAI\n",
    "os.environ[\"OPENAI_API_BASE\"] = os.environ[\"AZURE_OPENAI_ENDPOINT\"]\n",
    "os.environ[\"OPENAI_API_KEY\"] = os.environ[\"AZURE_OPENAI_API_KEY\"]\n",
    "os.environ[\"OPENAI_API_VERSION\"] = os.environ[\"AZURE_OPENAI_API_VERSION\"]\n",
    "os.environ[\"OPENAI_API_TYPE\"] = \"azure\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325d9138-2250-4f6b-bc88-50d7957f8d33",
   "metadata": {},
   "source": [
    "**Important Note**: Starting now, we will utilize OpenAI models. Please ensure that you have deployed the following models within the Azure OpenAI portal using these precise deployment names:\n",
    "\n",
    "- text-embedding-ada-002\n",
    "- gpt-35-turbo\n",
    "- gpt-35-turbo-16k\n",
    "- gpt-4\n",
    "- gpt-4-32k\n",
    "\n",
    "Should you have deployed the models under different names, the code provided below will not function as expected. To resolve this, you would need to modify the variable names throughout all the notebooks."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e7c720e-ece1-45ad-9d01-2dfd15c182bb",
   "metadata": {},
   "source": [
    "## A gentle intro to chaining LLMs and prompt engineering"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2bcd7028-5a6c-4296-8c85-4f420d408d69",
   "metadata": {},
   "source": [
    "Chains are what you get by connecting one or more large language models (LLMs) in a logical way. (Chains can be built of entities other than LLMs but for now, let’s stick with this definition for simplicity).\n",
    "\n",
    "Azure OpenAI is a type of LLM (provider) that you can use but there are others like Cohere, Huggingface, etc.\n",
    "\n",
    "Chains can be simple (i.e. Generic) or specialized (i.e. Utility).\n",
    "\n",
    "* Generic — A single LLM is the simplest chain. It takes an input prompt and the name of the LLM and then uses the LLM for text generation (i.e. output for the prompt).\n",
    "\n",
    "Here’s an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "13df9247-e784-4e04-9475-55e672efea47",
   "metadata": {},
   "outputs": [],
   "source": [
    "MODEL = \"gpt-35-turbo\" # options: gpt-35-turbo, gpt-35-turbo-16k, gpt-4, gpt-4-32k\n",
    "COMPLETION_TOKENS = 1000\n",
    "llm = AzureChatOpenAI(deployment_name=MODEL, temperature=0, max_tokens=COMPLETION_TOKENS)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7b0520b9-83b2-49fd-ad84-624cb0f15ce1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Answer the following question: \"Explain the difference between exploration and explotation\". Give your response in French\n"
     ]
    }
   ],
   "source": [
    "# Now we create a simple prompt template\n",
    "prompt = PromptTemplate(\n",
    "    input_variables=[\"question\", \"language\"],\n",
    "    template='Answer the following question: \"{question}\". Give your response in {language}',\n",
    ")\n",
    "\n",
    "print(prompt.format(question=QUESTION, language=\"French\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "dcc7dae3-6b88-4ea6-be43-b178ebc559dc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'Explain the difference between exploration and explotation',\n",
       " 'language': 'French',\n",
       " 'text': \"L'exploration et l'exploitation sont deux concepts distincts qui sont souvent utilisés dans des contextes différents.\\n\\nL'exploration se réfère à l'action d'explorer, de découvrir ou de rechercher de nouvelles opportunités, connaissances ou territoires. C'est un processus de découverte et d'expérimentation qui vise à trouver de nouvelles idées, de nouveaux marchés ou de nouvelles ressources. L'exploration implique souvent une certaine dose d'incertitude et de risque, car elle nécessite de sortir de sa zone de confort et d'essayer des choses nouvelles et inconnues.\\n\\nD'un autre côté, l'exploitation se concentre sur l'utilisation et la maximisation des ressources, des connaissances ou des opportunités déjà connues et maîtrisées. C'est le processus de tirer profit de ce qui est déjà connu et établi, en utilisant des méthodes et des stratégies éprouvées pour obtenir des résultats optimaux. L'exploitation est souvent associée à la consolidation, à l'efficacité et à la répétition de ce qui fonctionne déjà.\\n\\nEn résumé, l'exploration est axée sur la découverte de nouvelles possibilités et l'expérimentation, tandis que l'exploitation se concentre sur l'utilisation et la maximisation des ressources et des connaissances déjà acquises.\"}"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# And finnaly we create our first generic chain\n",
    "chain_chat = LLMChain(llm=llm, prompt=prompt)\n",
    "chain_chat({\"question\": QUESTION, \"language\": \"French\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cd8539d0-a538-4368-82c3-5f91d8370f1e",
   "metadata": {},
   "source": [
    "**Note**: this is the first time you use OpenAI in this Accelerator, so if you get a Resource not found error, is most likely because the name of your OpenAI model deployment is different than the variable MODEL set above"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50ed014c-0c6b-448c-b995-fe7970b92ad5",
   "metadata": {},
   "source": [
    "Great!!, now you know how to create a simple prompt and use a chain in order to answer a general question using ChatGPT knowledge!. \n",
    "\n",
    "It is important to note that we rarely use generic chains as standalone chains. More often they are used as building blocks for Utility chains (as we will see next). Also important to notice is that we are NOT using our documents or the result of the Azure Search yet, just the knowledge of ChatGPT on the data it was trained on."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12c48038-b1af-4228-8ffb-720e554fd3b2",
   "metadata": {
    "tags": []
   },
   "source": [
    "**The second type of Chains are Utility:**\n",
    "\n",
    "* Utility — These are specialized chains, comprised of many LLMs to help solve a specific task. For example, LangChain supports some end-to-end chains (such as [QA_WITH_SOURCES](https://python.langchain.com/en/latest/modules/chains/index_examples/qa_with_sources.html) for QnA Doc retrieval, Summarization, etc) and some specific ones (such as GraphQnAChain for creating, querying, and saving graphs). \n",
    "\n",
    "We will look at one specific chain called **qa_with_sources** in this workshop for digging deeper and solve our use case of enhancing the results of Azure Cognitive Search."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0454ddb-44d8-4fa9-929a-5e5563dd28f8",
   "metadata": {},
   "source": [
    "\n",
    "But before dealing with the utility chain needed, we need to deal first with this problem: **the content of the search result files is or can be very lengthy, more than the allowed tokens allowed by the GPT Azure OpenAI models**. \n",
    "\n",
    "This is where the concept of embeddings/vectors come into place.\n",
    "\n",
    "## Embeddings and Vector Search\n",
    "\n",
    "From the Azure OpenAI documentation ([HERE](https://learn.microsoft.com/en-us/azure/ai-services/openai/how-to/embeddings?tabs=python)), An embedding is a special format of data representation that can be easily utilized by machine learning models and algorithms. The embedding is an information dense representation of the semantic meaning of a piece of text. Each embedding is a vector of floating point numbers, such that the distance between two embeddings in the vector space is correlated with semantic similarity between two inputs in the original format. For example, if two texts are similar, then their vector representations should also be similar. \n",
    "\n",
    "To address the challenge of accommodating context within the token limit of a Language Model (LLM), the solution involves the following steps:\n",
    "\n",
    "1. **Segmenting Documents**: Divide the documents into smaller segments or chunks.\n",
    "2. **Vectorization of Chunks**: Transform these chunks into vectors using appropriate techniques.\n",
    "3. **Vector Semantic Search**: Execute a semantic search using vectors to identify the top chunks similar to the given question.\n",
    "4. **Optimal Context Provision**: Provide the LLM with the most relevant and concise context, thereby achieving an optimal balance between comprehensiveness and lengthiness.\n",
    "\n",
    "\n",
    "Notice that **the documents chunks are already done in Azure Search**. *ordered_content* dictionary (created a few cells above) contains the chunks of each document. So we don't really need to chunk them again, but we still need to make sure that we can be as fast as possible and that we are below the max allowed input token limits of our selected OpenAI model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "80e79235-3d8b-4713-9336-5004cc4a1556",
   "metadata": {},
   "source": [
    "Our ultimate goal is to rely solely on vector indexes. While it is possible to manually code parsers with OCR for various file types and develop a scheduler to synchronize data with the index, there is a more efficient alternative: **Azure Cognitive Search is soon going to release automated chunking strategies and vectorization within the next months**, so we have three options: \n",
    "1. Wait for this functionality while in the meantime manually push chunks and its vectors to the vector-based indexes \n",
    "2. Fill up the vector-based indexes on-demand, as documents are discovered by users\n",
    "3. Use custom skills (for chunking and vectorization) and use knowledge stores in order to create a vector-base index from a text-based-ai-enriched index at ingestion time. See [HERE](https://github.com/Azure/cognitive-search-vector-pr/blob/main/demo-python/code/azure-search-vector-ingestion-python-sample.ipynb) for instructions on how to do this.\n",
    "\n",
    "In this notebook we are going to implement Option 2: **Create vector-based indexes per each text-based indexes and fill them up on-demand as documents are discovered**. Why? because is simpler and quick to implement, while we wait for Option 1 to become a feature of Azure Search Engine (which is the automation of Option 3 inside the search engine).\n",
    "\n",
    "As observed in Notebooks 1 and 2, each text-based index contains a field named `vectorized` that we have not utilized yet. We will now harness this field. The objective is to avoid vectorizing all documents at the time of ingestion (Option 3). Instead, we can vectorize the chunks as users search for or discover documents. This approach ensures that we allocate funds and resources only when the documents are actually required. Typically, in an organization with a vast repository of documents in a data lake, only 20% of the documents are frequently accessed, while the rest remain untouched. This phenomenon mirrors the [Pareto Principle](https://en.wikipedia.org/wiki/Pareto_principle) found in nature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "12682a1b-df92-49ce-a638-7277103f6cb3",
   "metadata": {},
   "outputs": [],
   "source": [
    "index_name = \"cogsrch-index-files\"\n",
    "index2_name = \"cogsrch-index-csv\"\n",
    "indexes = [index_name, index2_name]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78a6d6a7-18ef-45b2-a216-3c1f50006593",
   "metadata": {},
   "source": [
    "In order to not duplicate code, we have put many of the code used above into functions. These functions are in the `common/utils.py` and `common/prompts.py` files. This way we can use these functios in the app that we will build later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "3bccca45-d1dd-476f-b109-a528b857b6b3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 16\n"
     ]
    }
   ],
   "source": [
    "k = 10 # Number of results per each text_index\n",
    "ordered_results = get_search_results(QUESTION, indexes, k=10, reranker_threshold=1)\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "7714f38a-daaa-4fc5-a95a-dd025d153216",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment the below line if you want to inspect the ordered results\n",
    "# ordered_results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "da70e7a8-7536-4688-b30c-01ba28e9b9f8",
   "metadata": {},
   "source": [
    "Now we can fill up the vector-based index as users lookup documents using the text-based index. This approach although it requires two searches per user query (one on the text-based indexes and the other one on the vector-based indexes), it is simpler to implement and will be incrementatly faster as user use the system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "2937ba3b-098d-43f8-8498-3534882a5cc7",
   "metadata": {},
   "outputs": [],
   "source": [
    "embedder = OpenAIEmbeddings(deployment=\"text-embedding-ada-002\", chunk_size=1) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f664df30-99c3-4a30-8cb0-42ba3044e5b0",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206262/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7135900/\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1016/j.ejmech.2016.11.027; https://www.ncbi.nlm.nih.gov/pubmed/27914364/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3594938/\n",
      "Vectorizing 52 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/0006/0006012v1.pdf\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1017/s0950268808001544; https://www.ncbi.nlm.nih.gov/pubmed/19017431/\n",
      "Vectorizing 1 chunks from Document: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1160249/\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1016/j.ciresp.2013.02.010; https://www.ncbi.nlm.nih.gov/pubmed/24559592/\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1016/j.ijsu.2016.10.012; https://www.ncbi.nlm.nih.gov/pubmed/27743897/\n",
      "Vectorizing 8 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/0012/0012003v1.pdf\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1016/j.surg.2014.06.020; https://www.ncbi.nlm.nih.gov/pubmed/25239339/\n",
      "Vectorizing 1 chunks from Document: https://doi.org/10.1111/vsu.12693; https://www.ncbi.nlm.nih.gov/pubmed/28906566/\n",
      "Vectorizing 9 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/0007/0007001v1.pdf\n",
      "Vectorizing 2 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/0003/0003057v1.pdf\n",
      "Vectorizing 35 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/0004/0004001v1.pdf\n",
      "Vectorizing 73 chunks from Document: https://demodatasetsp.blob.core.windows.net/arxivcs/0004/0004014v2.pdf\n",
      "CPU times: user 8.98 s, sys: 185 ms, total: 9.17 s\n",
      "Wall time: 42.2 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "for key,value in ordered_results.items():\n",
    "    if value[\"vectorized\"] != True: # If the document has not been vectorized yet\n",
    "        i = 0\n",
    "        print(\"Vectorizing\",len(value[\"chunks\"]),\"chunks from Document:\",value[\"location\"])\n",
    "        for chunk in value[\"chunks\"]: # Iterate over the document's text chunks\n",
    "            try:\n",
    "                upload_payload = {  # Insert the chunk and its vector in the vector-based index\n",
    "                    \"value\": [\n",
    "                        {\n",
    "                            \"id\": key + \"_\" + str(i),\n",
    "                            \"title\": f\"{value['title']}_chunk_{str(i)}\",\n",
    "                            \"chunk\": chunk,\n",
    "                            \"chunkVector\": embedder.embed_query(chunk if chunk!=\"\" else \"-------\"),\n",
    "                            \"name\": value[\"name\"],\n",
    "                            \"location\": value[\"location\"],\n",
    "                            \"@search.action\": \"upload\"\n",
    "                        },\n",
    "                    ]\n",
    "                }\n",
    "\n",
    "                r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + value[\"index\"]+\"-vector\" + \"/docs/index\",\n",
    "                                     data=json.dumps(upload_payload), headers=headers, params=params)\n",
    "                \n",
    "                if r.status_code != 200:\n",
    "                    print(r.status_code)\n",
    "                    print(r.text)\n",
    "                else:\n",
    "                    i = i + 1 # increment chunk number\n",
    "                    \n",
    "            except Exception as e:\n",
    "                print(\"Exception:\",e)\n",
    "                print(chunk)\n",
    "                continue\n",
    "                    \n",
    "        # Update document in text-based index and mark it as \"vectorized\"\n",
    "        upload_payload = {\n",
    "            \"value\": [\n",
    "                {\n",
    "                    \"id\": key,\n",
    "                    \"vectorized\": True,\n",
    "                    \"@search.action\": \"merge\"\n",
    "                },\n",
    "            ]\n",
    "        }\n",
    "\n",
    "        r = requests.post(os.environ['AZURE_SEARCH_ENDPOINT'] + \"/indexes/\" + value[\"index\"]+ \"/docs/index\",\n",
    "                         data=json.dumps(upload_payload), headers=headers, params=params)\n",
    "                    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f490b7fe-eec2-4c96-a2f2-f8ab0a1b2098",
   "metadata": {},
   "source": [
    "**Note**: How the text-based and the vector-based indexes stay in sync?\n",
    "For document changes, the problem is already taken care of, since Azure Engine will update the text-based index automatically if a file has a new version. This puts the vectorized field in None and the next time that the file is searched it will be vectorized again into the vector-based index.\n",
    "\n",
    "However for deletion of files, the problem is half solved. Azure Search engine would delete the documents in the text-based index if the file is deleted on the source, however you will need to code a script that runs on a fixed schedule that looks for deleted ids in the text-based index and deletes the corresponding chunks in the vector-based index."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1f67f3a2-0023-4f5a-b52f-3fb071cfd8e1",
   "metadata": {},
   "source": [
    "Now we search on the vector-based indexes and get the top k most similar chunks to our question:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "61098bb4-33da-4eb4-94cf-503587337aca",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of results: 3\n"
     ]
    }
   ],
   "source": [
    "vector_indexes = [index+\"-vector\" for index in indexes]\n",
    "\n",
    "k = 10\n",
    "similarity_k = 3\n",
    "ordered_results = get_search_results(QUESTION, vector_indexes,\n",
    "                                        k=k, # Number of results per vector index\n",
    "                                        reranker_threshold=1,\n",
    "                                        vector_search=True, \n",
    "                                        similarity_k=similarity_k,\n",
    "                                        query_vector = embedder.embed_query(QUESTION)\n",
    "                                        )\n",
    "print(\"Number of results:\",len(ordered_results))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a98a974-0633-499f-a8f0-29bf6242e737",
   "metadata": {},
   "source": [
    "For vector search is not recommended to give more than k=5 chunks (of max 5000 characters each) to the LLM as context. Otherwise you can have issues later with the token limit trying to have a conversation with memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "7dfb9e39-2542-469d-8f64-4c0c26d79535",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of chunks: 3\n"
     ]
    }
   ],
   "source": [
    "top_docs = []\n",
    "for key,value in ordered_results.items():\n",
    "    location = value[\"location\"] if value[\"location\"] is not None else \"\"\n",
    "    top_docs.append(Document(page_content=value[\"chunk\"], metadata={\"source\": location}))\n",
    "        \n",
    "print(\"Number of chunks:\",len(top_docs))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "880885fe-16bd-44bb-9556-7cb3d4989993",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "System prompt token count: 1669\n",
      "Max Completion Token count: 1000\n",
      "Combined docs (context) token count: 628\n",
      "--------\n",
      "Requested token count: 3297\n",
      "Token limit for gpt-35-turbo : 4096\n",
      "Chain Type selected: stuff\n"
     ]
    }
   ],
   "source": [
    "# Calculate number of tokens of our docs\n",
    "if(len(top_docs)>0):\n",
    "    tokens_limit = model_tokens_limit(MODEL) # this is a custom function we created in common/utils.py\n",
    "    prompt_tokens = num_tokens_from_string(COMBINE_PROMPT_TEMPLATE) # this is a custom function we created in common/utils.py\n",
    "    context_tokens = num_tokens_from_docs(top_docs) # this is a custom function we created in common/utils.py\n",
    "    \n",
    "    requested_tokens = prompt_tokens + context_tokens + COMPLETION_TOKENS\n",
    "    \n",
    "    chain_type = \"map_reduce\" if requested_tokens > 0.9 * tokens_limit else \"stuff\"  \n",
    "    \n",
    "    print(\"System prompt token count:\",prompt_tokens)\n",
    "    print(\"Max Completion Token count:\", COMPLETION_TOKENS)\n",
    "    print(\"Combined docs (context) token count:\",context_tokens)\n",
    "    print(\"--------\")\n",
    "    print(\"Requested token count:\",requested_tokens)\n",
    "    print(\"Token limit for\", MODEL, \":\", tokens_limit)\n",
    "    print(\"Chain Type selected:\", chain_type)\n",
    "        \n",
    "else:\n",
    "    print(\"NO RESULTS FROM AZURE SEARCH\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e232424-c7ba-4153-b23b-fb1fa2ebc64b",
   "metadata": {},
   "source": [
    "Now we will use our Utility Chain from LangChain `qa_with_sources`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "511273b3-256d-4e60-be72-ccd4a74cb885",
   "metadata": {},
   "outputs": [],
   "source": [
    "if chain_type == \"stuff\":\n",
    "    chain = load_qa_with_sources_chain(llm, chain_type=chain_type, \n",
    "                                       prompt=COMBINE_PROMPT)\n",
    "elif chain_type == \"map_reduce\":\n",
    "    chain = load_qa_with_sources_chain(llm, chain_type=chain_type, \n",
    "                                       question_prompt=COMBINE_QUESTION_PROMPT,\n",
    "                                       combine_prompt=COMBINE_PROMPT,\n",
    "                                       return_intermediate_steps=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "b99a0c19-d48c-41e9-8d6c-6d9f13d29da3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 7.97 ms, sys: 4 µs, total: 7.97 ms\n",
      "Wall time: 7.18 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "# Try with other language as well\n",
    "response = chain({\"input_documents\": top_docs, \"question\": QUESTION, \"language\": \"English\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "37f7fa67-f67b-402e-89e3-266d5d6d21d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/markdown": [
       "Exploration and exploitation are two different approaches in reinforcement learning. Exploration refers to the process of actively seeking out new and unfamiliar states or actions in order to gather more information about the environment. It involves taking actions that may not yield immediate rewards but have the potential to discover new knowledge. On the other hand, exploitation involves utilizing the existing knowledge or information to maximize the rewards or benefits. It focuses on taking actions that are known to be effective based on previous experiences.\n",
       "\n",
       "In the context of the first content, the paper discusses the challenge of sparse reward tasks in reinforcement learning and the need for both exploration and exploitation. It introduces two different approaches: self-imitation learning, which emphasizes exploitation by imitating past good trajectories, and exploration bonuses, which enhance exploration by providing intrinsic rewards for visiting novel states. The Explore-then-Exploit (EE) framework is then proposed, which combines these two approaches to strengthen their effects and achieve superior performance in MuJoCo environments with episodic reward settings. [1]\n",
       "\n",
       "In the context of the second content, the term \"exploit\" is used in a different context. It refers to how enveloped viruses take advantage of the existing routes of membrane traffic to enter and leave host cells. The similarity between viral envelopes and cellular membranes allows animal viruses to exploit these routes and use them for their own purposes. This exploitation of cellular mechanisms is used as a probe to study aspects of intracellular traffic. [2]\n",
       "\n",
       "In the context of the third content, the term \"exploration\" is used in the context of laparoscopic hernia repair. The study compares the occurrence of contralateral metachronous inguinal hernia (CMIH) after laparoscopic total extraperitoneal (TEP) repair with or without contralateral exploration. Contralateral exploration refers to the exploration of the asymptomatic contralateral inguinal hernia during the surgical procedure. The study found that simultaneous exploration and repair of the incidental defects on the contralateral inguinal region during laparoscopic TEP repair of unilateral inguinal hernia can effectively prevent later CMIH. [3]\n",
       "\n",
       "References:\n",
       "[1] Source: <sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7206262/\" target=\"_blank\">[1]</a></sup>\n",
       "[2] Source: <sup><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7135900/\" target=\"_blank\">[2]</a></sup>\n",
       "[3] Source: <sup><a href=\"https://doi.org/10.1016/j.ijsu.2016.10.012; https://www.ncbi.nlm.nih.gov/pubmed/27743897/\" target=\"_blank\">[3]</a></sup>"
      ],
      "text/plain": [
       "<IPython.core.display.Markdown object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "display(Markdown(response['output_text']))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05e27c75-bfd9-4304-b2fd-c8e30bcc0558",
   "metadata": {},
   "source": [
    "**Please Note**: There are some instances where, despite the answer's high accuracy and quality, the references are not done according to the instructions provided in the COMBINE_PROMPT. This behavior is anticipated when dealing with GPT-3.5 models. We will provide a more detailed explanation of this phenomenon towards the conclusion of Notebook 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "11345374-6420-4b36-b061-795d2a804c85",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment if you want to inspect the results from map_reduce chain type, each top similar chunk summary (k=4 by default)\n",
    "\n",
    "# if chain_type == \"map_reduce\":\n",
    "#     for step in response['intermediate_steps']:\n",
    "#         display(HTML(\"<b>Chunk Summary:</b> \" + step))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f347373a-a5be-473d-b64e-0f6b6dbcd0e0",
   "metadata": {},
   "source": [
    "# Summary\n",
    "##### This answer is way better than taking just the result from Azure Cognitive Search. So the summary is:\n",
    "- Utilizing Azure Cognitive Search, we conduct a multi-index text-based search that identifies the top documents from each index.\n",
    "- Utilizing Azure Cognitive Search's vector search, we extract the most relevant chunks of information.\n",
    "- Subsequently, Azure OpenAI utilizes these extracted chunks as context, comprehends the content, and employs it to deliver optimal answers.\n",
    "- Best of two worlds!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fdc6e2fe-1c34-4952-99ad-14940f022379",
   "metadata": {},
   "source": [
    "# NEXT\n",
    "In the next notebook, we are going to see how we can treat complex and large documents separately, also using Vector Search"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.10 - SDK v2",
   "language": "python",
   "name": "python310-sdkv2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}