langchain_redis.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Semantic Search with Redis LangChain & OpenAI"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Redis (Remote Dictionary Server) is an in-memory data structure store, used as a distributed, in-memory key–value database, cache and message broker, with optional durability."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pip install redis\n",
    "# pip install pypdf"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Import libraries and set API Key"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import openai\n",
    "import getpass\n",
    "from langchain.llms import OpenAI\n",
    "from langchain.chains import RetrievalQA\n",
    "from langchain.vectorstores.redis import Redis\n",
    "from langchain.document_loaders import TextLoader\n",
    "from langchain.embeddings import OpenAIEmbeddings\n",
    "from langchain.chains import ConversationalRetrievalChain\n",
    "from langchain.text_splitter import CharacterTextSplitter\n",
    "from langchain.chains.question_answering import load_qa_chain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "p = getpass.getpass(prompt = '\\nPlease enter your OpenAI API Secret Key (Input hidden):') \n",
    "\n",
    "os.environ['OPENAI_API_KEY'] = p\n",
    "openai.api_key = os.environ[\"OPENAI_API_KEY\"]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Load Documents file .txt"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.1. Load file .txt and Embedding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "loader = TextLoader('state_of_the_union.txt')\n",
    "documents = loader.load()\n",
    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "docs = text_splitter.split_documents(documents)\n",
    "\n",
    "embeddings = OpenAIEmbeddings()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \\n\\nLast year COVID-19 kept us apart. This year we are finally together again. \\n\\nTonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \\n\\nWith a duty to one another to the American people to the Constitution. \\n\\nAnd with an unwavering resolve that freedom will always triumph over tyranny. \\n\\nSix days ago, Russia’s Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \\n\\nHe thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \\n\\nHe met the Ukrainian people. \\n\\nFrom President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world.', metadata={'source': 'state_of_the_union.txt'}),\n",
       " Document(page_content='Groups of citizens blocking tanks with their bodies. Everyone from students to retirees teachers turned soldiers defending their homeland. \\n\\nIn this struggle as President Zelenskyy said in his speech to the European Parliament “Light will win over darkness.” The Ukrainian Ambassador to the United States is here tonight. \\n\\nLet each of us here tonight in this Chamber send an unmistakable signal to Ukraine and to the world. \\n\\nPlease rise if you are able and show that, Yes, we the United States of America stand with the Ukrainian people. \\n\\nThroughout our history we’ve learned this lesson when dictators do not pay a price for their aggression they cause more chaos.   \\n\\nThey keep moving.   \\n\\nAnd the costs and the threats to America and the world keep rising.   \\n\\nThat’s why the NATO Alliance was created to secure peace and stability in Europe after World War 2. \\n\\nThe United States is a member along with 29 other nations. \\n\\nIt matters. American diplomacy matters. American resolve matters.', metadata={'source': 'state_of_the_union.txt'}),\n",
       " Document(page_content='Putin’s latest attack on Ukraine was premeditated and unprovoked. \\n\\nHe rejected repeated efforts at diplomacy. \\n\\nHe thought the West and NATO wouldn’t respond. And he thought he could divide us at home. Putin was wrong. We were ready.  Here is what we did.   \\n\\nWe prepared extensively and carefully. \\n\\nWe spent months building a coalition of other freedom-loving nations from Europe and the Americas to Asia and Africa to confront Putin. \\n\\nI spent countless hours unifying our European allies. We shared with the world in advance what we knew Putin was planning and precisely how he would try to falsely justify his aggression.  \\n\\nWe countered Russia’s lies with truth.   \\n\\nAnd now that he has acted the free world is holding him accountable. \\n\\nAlong with twenty-seven members of the European Union including France, Germany, Italy, as well as countries like the United Kingdom, Canada, Japan, Korea, Australia, New Zealand, and many others, even Switzerland.', metadata={'source': 'state_of_the_union.txt'})]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "docs[:3]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.2. Connect Redis DB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'link'"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rds = Redis.from_documents(docs, embeddings, redis_url=\"redis://localhost:6379\",  index_name='link')\n",
    "rds.index_name"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.3. Semantic search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
      "\n",
      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
      "\n",
      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
      "\n",
      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.\n"
     ]
    }
   ],
   "source": [
    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
    "results = rds.similarity_search(query)\n",
    "print(results[0].page_content)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Call OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_similiar_docs(query, k=2, score=False):\n",
    "  if score:\n",
    "    similar_docs = rds.similarity_search_with_score(query, k=k)\n",
    "  else:\n",
    "    similar_docs = rds.similarity_search(query, k=k)\n",
    "  return similar_docs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [],
   "source": [
    "llm = OpenAI(model_name=\"text-davinci-003\", temperature=0)\n",
    "\n",
    "chain = load_qa_chain(llm, chain_type=\"stuff\")\n",
    "\n",
    "def get_answer(query):\n",
    "  similar_docs = get_similiar_docs(query)\n",
    "  answer = chain.run(input_documents=similar_docs, question=query)\n",
    "  return answer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " The president said that Ketanji Brown Jackson is one of our nation's top legal minds, a former top litigator in private practice, a former federal public defender, and from a family of public school educators and police officers. He also said that she is a consensus builder and has received a broad range of support from the Fraternal Order of Police to former judges appointed by Democrats and Republicans.\n"
     ]
    }
   ],
   "source": [
    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
    "answer = get_answer(query)\n",
    "print(answer)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.4. Add texts on Document"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['doc:link:a42838898c0745d19ee4dee1624638c6']\n"
     ]
    }
   ],
   "source": [
    "print(rds.add_texts([\"Ankush went to Princeton\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ankush went to Princeton\n"
     ]
    }
   ],
   "source": [
    "query = \"Princeton\"\n",
    "results = rds.similarity_search(query)\n",
    "print(results[0].page_content)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.5. Load from existing index"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. \n",
      "\n",
      "Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. \n",
      "\n",
      "One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. \n",
      "\n",
      "And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.\n"
     ]
    }
   ],
   "source": [
    "# Load from existing index\n",
    "rds = Redis.from_existing_index(embeddings, redis_url=\"redis://localhost:6379\", index_name='link')\n",
    "\n",
    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
    "results = rds.similarity_search(query)\n",
    "print(results[0].page_content)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2.6. RedisVectorStoreRetriever"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Here we go over different options for using the vector store as a retriever.\n",
    "\n",
    "* There are three different search methods we can use to do retrieval. By default, it will use semantic similarity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "retriever = rds.as_retriever()\n",
    "docs = retriever.get_relevant_documents(query)\n",
    "retriever = rds.as_retriever(search_type=\"similarity_limit\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='Ankush went to Princeton', metadata={}),\n",
       " Document(page_content='One was stationed at bases and breathing in toxic smoke from “burn pits” that incinerated wastes of war—medical and hazard material, jet fuel, and more. \\n\\nWhen they came home, many of the world’s fittest and best trained warriors were never the same. \\n\\nHeadaches. Numbness. Dizziness. \\n\\nA cancer that would put them in a flag-draped coffin. \\n\\nI know. \\n\\nOne of those soldiers was my son Major Beau Biden. \\n\\nWe don’t know for sure if a burn pit was the cause of his brain cancer, or the diseases of so many of our troops. \\n\\nBut I’m committed to finding out everything we can. \\n\\nCommitted to military families like Danielle Robinson from Ohio. \\n\\nThe widow of Sergeant First Class Heath Robinson.  \\n\\nHe was born a soldier. Army National Guard. Combat medic in Kosovo and Iraq. \\n\\nStationed near Baghdad, just yards from burn pits the size of football fields. \\n\\nHeath’s widow Danielle is here with us tonight. They loved going to Ohio State football games. He loved building Legos with their daughter.', metadata={'source': 'state_of_the_union.txt'}),\n",
       " Document(page_content='Tonight, I’m announcing a crackdown on these companies overcharging American businesses and consumers. \\n\\nAnd as Wall Street firms take over more nursing homes, quality in those homes has gone down and costs have gone up.  \\n\\nThat ends on my watch. \\n\\nMedicare is going to set higher standards for nursing homes and make sure your loved ones get the care they deserve and expect. \\n\\nWe’ll also cut costs and keep the economy going strong by giving workers a fair shot, provide more training and apprenticeships, hire them based on their skills not degrees. \\n\\nLet’s pass the Paycheck Fairness Act and paid leave.  \\n\\nRaise the minimum wage to $15 an hour and extend the Child Tax Credit, so no one has to raise a family in poverty. \\n\\nLet’s increase Pell Grants and increase our historic support of HBCUs, and invest in what Jill—our First Lady who teaches full-time—calls America’s best-kept secret: community colleges.', metadata={'source': 'state_of_the_union.txt'}),\n",
       " Document(page_content='If you travel 20 miles east of Columbus, Ohio, you’ll find 1,000 empty acres of land. \\n\\nIt won’t look like much, but if you stop and look closely, you’ll see a “Field of dreams,” the ground on which America’s future will be built. \\n\\nThis is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor “mega site”. \\n\\nUp to eight state-of-the-art factories in one place. 10,000 new good-paying jobs. \\n\\nSome of the most sophisticated manufacturing in the world to make computer chips the size of a fingertip that power the world and our everyday lives. \\n\\nSmartphones. The Internet. Technology we have yet to invent. \\n\\nBut that’s just the beginning. \\n\\nIntel’s CEO, Pat Gelsinger, who is here tonight, told me they are ready to increase their investment from  \\n$20 billion to $100 billion. \\n\\nThat would be one of the biggest investments in manufacturing in American history. \\n\\nAnd all they’re waiting for is for you to pass this bill.', metadata={'source': 'state_of_the_union.txt'})]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here we can see it doesn't return any results because there are no relevant documents\n",
    "retriever.get_relevant_documents(\"where did ankush go to college?\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### RetrievalQA with OpenAI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "result  {'query': 'What did the president say about Ketanji Brown Jackson', 'result': \" The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\"}\n"
     ]
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(llm=OpenAI(model_name=\"text-davinci-003\"), retriever=rds.as_retriever())\n",
    "\n",
    "question = \"What did the president say about Ketanji Brown Jackson\"\n",
    "\n",
    "result = qa({\"query\": question})\n",
    "print(\"result \", result)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### ConversationalRetrievalChain"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.vectorstores import Chroma\n",
    "from langchain.document_loaders import PyPDFLoader\n",
    "\n",
    "# load document\n",
    "loader = PyPDFLoader(\"./2023_GPT4All_Technical_Report.pdf\")\n",
    "documents = loader.load()\n",
    "# split the documents into chunks\n",
    "text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "texts = text_splitter.split_documents(documents)\n",
    "# select which embeddings we want to use\n",
    "embeddings = OpenAIEmbeddings()\n",
    "# create the vectorestore to use as the index\n",
    "rds = Redis.from_documents(texts, embeddings, redis_url=\"redis://localhost:6379\",  index_name='link')\n",
    "\n",
    "# expose this index in a retriever interface\n",
    "retriever = rds.as_retriever(search_type=\"similarity\", search_kwargs={\"k\":2})\n",
    "# create a chain to answer questions \n",
    "qa = ConversationalRetrievalChain.from_llm(OpenAI(model_name=\"text-davinci-003\"), retriever)\n",
    "chat_history = []\n",
    "query = \"what is the total number of AI publications?\"\n",
    "result = qa({\"question\": query, \"chat_history\": chat_history})\n",
    "chat_history.append({'query':query, 'answer':result[\"answer\"]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'what is the total number of AI publications?',\n",
       " 'chat_history': [{'query': 'what is the total number of AI publications?',\n",
       "   'answer': \" I don't know.\"}],\n",
       " 'answer': \" I don't know.\"}"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"What did the president say about Ketanji Brown Jackson\"\n",
    "result = qa({\"question\": query, \"chat_history\": chat_history})\n",
    "chat_history.append({'query':query, 'answer':result[\"answer\"]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'question': 'What did the president say about Ketanji Brown Jackson',\n",
       " 'chat_history': [{'query': 'what is the total number of AI publications?',\n",
       "   'answer': \" I don't know.\"},\n",
       "  {'query': 'What did the president say about Ketanji Brown Jackson',\n",
       "   'answer': \" The President said that Ketanji Brown Jackson is one of the nation's top legal minds, and that she will continue Justice Breyer's legacy of excellence.\"}],\n",
       " 'answer': \" The President said that Ketanji Brown Jackson is one of the nation's top legal minds, and that she will continue Justice Breyer's legacy of excellence.\"}"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('what is the total number of AI publications?', \" I don't know.\")]"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "chat_history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "\" The president said that Ketanji Brown Jackson is one of the nation's top legal minds and that she will continue Justice Breyer's legacy of excellence.\""
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result['answer']"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Load Document file .pdf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [],
   "source": [
    "from langchain.document_loaders import PyPDFLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load document\n",
    "loader_pdf = PyPDFLoader(\"./2023_GPT4All_Technical_Report.pdf\")\n",
    "documents_pdf = loader_pdf.load()\n",
    "\n",
    "# split the documents into chunks\n",
    "text_splitter_pdf = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)\n",
    "texts_pdf = text_splitter_pdf.split_documents(documents_pdf)\n",
    "# select which embeddings we want to use\n",
    "embeddings_pdf = OpenAIEmbeddings()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'link'"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rds_pdf = Redis.from_documents(documents_pdf, embeddings_pdf, redis_url=\"redis://localhost:6379\",  index_name='link')\n",
    "rds_pdf.index_name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# expose this index in a retriever interface\n",
    "retriever = rds_pdf.as_retriever(search_type=\"similarity\", search_kwargs={\"k\":2})\n",
    "# create a chain to answer questions \n",
    "qa = RetrievalQA.from_chain_type(\n",
    "    llm=OpenAI(), chain_type=\"stuff\", retriever=retriever, return_source_documents=True)\n",
    "query = \"what is the total number of AI publications?\"\n",
    "result = qa({\"query\": query})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [],
   "source": [
    "query = \"What was the cost of training the GPT4all model?\"\n",
    "retriever = rds_pdf.as_retriever()\n",
    "docs = retriever.get_relevant_documents(query)\n",
    "retriever = rds.as_retriever(search_type=\"similarity_limit\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='(a) TSNE visualization of the final training data, ten-colored\\nby extracted topic.\\n(b) Zoomed in view of Figure 2a. The region displayed con-\\ntains generations related to personal health and wellness.\\nFigure 2: The final training data was curated to ensure a diverse distribution of prompt topics and model responses.\\n2.1 Reproducibility\\nWe release all data (including unused P3 genera-\\ntions), training code, and model weights for the\\ncommunity to build upon. Please check the Git\\nrepository for the most up-to-date data, training\\ndetails and checkpoints.\\n2.2 Costs\\nWe were able to produce these models with about\\nfour days work, $800 in GPU costs (rented from\\nLambda Labs and Paperspace) including several\\nfailed trains, and $500 in OpenAI API spend.\\nOur released model, gpt4all-lora, can be trained in\\nabout eight hours on a Lambda Labs DGX A100\\n8x 80GB for a total cost of $100 .\\n3 Evaluation\\nWe perform a preliminary evaluation of our model\\nusing the human evaluation data from the Self-\\nInstruct paper (Wang et al., 2022). We report the\\nground truth perplexity of our model against what\\nis, to our knowledge, the best openly available\\nalpaca-lora model, provided by user chainyo on\\nhuggingface. We find that all models have very\\nlarge perplexities on a small number of tasks, and\\nreport perplexities clipped to a maximum of 100.\\nModels finetuned on this collected dataset ex-\\nhibit much lower perplexity in the Self-Instruct\\nevaluation compared to Alpaca. This evaluation is\\nin no way exhaustive and further evaluation work\\nFigure 3: Model Perplexities. Lower is better. Our\\nmodels achieve stochastically lower ground truth per-\\nplexities than alpaca-lora.\\nremains. We welcome the reader to run the model\\nlocally on CPU (see Github for files) and get a\\nqualitative sense of what it can do.\\n4 Use Considerations\\nThe authors release data and training details in\\nhopes that it will accelerate open LLM research,\\nparticularly in the domains of alignment and inter-\\npretability. GPT4All model weights and data are\\nintended and licensed only for research purposes\\nand any commercial use is prohibited. GPT4All\\nis based on LLaMA, which has a non-commercial\\nlicense. The assistant data is gathered from Ope-\\nnAI’s GPT-3.5-Turbo, whose terms of use pro-', metadata={'source': './2023_GPT4All_Technical_Report.pdf', 'page': 1}),\n",
       " Document(page_content='GPT4All: Training an Assistant-style Chatbot with Large Scale Data\\nDistillation from GPT-3.5-Turbo\\nYuvanesh Anand\\nyuvanesh@nomic.aiZach Nussbaum\\nzanussbaum@gmail.com\\nBrandon Duderstadt\\nbrandon@nomic.aiBenjamin Schmidt\\nben@nomic.aiAndriy Mulyar\\nandriy@nomic.ai\\nAbstract\\nThis preliminary technical report describes the\\ndevelopment of GPT4All, a chatbot trained\\nover a massive curated corpus of assistant in-\\nteractions including word problems, story de-\\nscriptions, multi-turn dialogue, and code. We\\nopenly release the collected data, data cura-\\ntion procedure, training code, and final model\\nweights to promote open research and repro-\\nducibility. Additionally, we release quantized\\n4-bit versions of the model allowing virtually\\nanyone to run the model on CPU.\\n1 Data Collection and Curation\\nWe collected roughly one million prompt-\\nresponse pairs using the GPT-3.5-Turbo OpenAI\\nAPI between March 20, 2023 and March 26th,\\n2023. To do this, we first gathered a diverse sam-\\nple of questions/prompts by leveraging three pub-\\nlicly available datasets:\\n• The unified chip2 subset of LAION OIG.\\n• Coding questions with a random sub-sample\\nof Stackoverflow Questions\\n• Instruction-tuning with a sub-sample of Big-\\nscience/P3\\nWe chose to dedicate substantial attention to data\\npreparation and curation based on commentary in\\nthe Stanford Alpaca project (Taori et al., 2023).\\nUpon collection of the initial dataset of prompt-\\ngeneration pairs, we loaded data into Atlas for data\\ncuration and cleaning. With Atlas, we removed all\\nexamples where GPT-3.5-Turbo failed to respond\\nto prompts and produced malformed output. This\\nreduced our total number of examples to 806,199\\nhigh-quality prompt-generation pairs. Next, we\\ndecided to remove the entire Bigscience/P3 sub-\\nset from the final training dataset due to its very\\nFigure 1: TSNE visualization of the candidate training\\ndata (Red: Stackoverflow, Orange: chip2, Blue: P3).\\nThe large blue balls (e.g. indicated by the red arrow)\\nare highly homogeneous prompt-response pairs.\\nlow output diversity; P3 contains many homoge-\\nneous prompts which produce short and homoge-\\nneous responses from GPT-3.5-Turbo. This exclu-\\nsion produces a final subset containing 437,605\\nprompt-generation pairs, which is visualized in\\nFigure 2. You can interactively explore the dataset\\nat each stage of cleaning at the following links:\\n• Cleaned with P3\\n• Cleaned without P3 (Final Training Dataset)\\n2 Model Training\\nWe train several models finetuned from an in-\\nstance of LLaMA 7B (Touvron et al., 2023).\\nThe model associated with our initial public re-\\nlease is trained with LoRA (Hu et al., 2021)\\non the 437,605 post-processed examples for four\\nepochs. Detailed model hyper-parameters and\\ntraining code can be found in the associated repos-\\nitory and model training log.', metadata={'source': './2023_GPT4All_Technical_Report.pdf', 'page': 0}),\n",
       " Document(page_content='hibit developing models that compete commer-\\ncially with OpenAI.\\nReferences\\nEdward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan\\nAllen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and\\nWeizhu Chen. 2021. Lora: Low-rank adaptation of\\nlarge language models.\\nRohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann\\nDubois, Xuechen Li, Carlos Guestrin, Percy\\nLiang, and Tatsunori B. Hashimoto. 2023. Stan-\\nford alpaca: An instruction-following llama\\nmodel. https://github.com/tatsu-lab/\\nstanford_alpaca .\\nHugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier\\nMartinet, Marie-Anne Lachaux, Timoth ´ee Lacroix,\\nBaptiste Rozi `ere, Naman Goyal, Eric Hambro,\\nFaisal Azhar, Aurelien Rodriguez, Armand Joulin,\\nEdouard Grave, and Guillaume Lample. 2023.\\nLlama: Open and efficient foundation language\\nmodels.\\nYizhong Wang, Yeganeh Kordi, Swaroop Mishra, Al-\\nisa Liu, Noah A. Smith, Daniel Khashabi, and Han-\\nnaneh Hajishirzi. 2022. Self-instruct: Aligning lan-\\nguage model with self generated instructions.', metadata={'source': './2023_GPT4All_Technical_Report.pdf', 'page': 2}),\n",
       " Document(page_content='If you travel 20 miles east of Columbus, Ohio, you’ll find 1,000 empty acres of land. \\n\\nIt won’t look like much, but if you stop and look closely, you’ll see a “Field of dreams,” the ground on which America’s future will be built. \\n\\nThis is where Intel, the American company that helped build Silicon Valley, is going to build its $20 billion semiconductor “mega site”. \\n\\nUp to eight state-of-the-art factories in one place. 10,000 new good-paying jobs. \\n\\nSome of the most sophisticated manufacturing in the world to make computer chips the size of a fingertip that power the world and our everyday lives. \\n\\nSmartphones. The Internet. Technology we have yet to invent. \\n\\nBut that’s just the beginning. \\n\\nIntel’s CEO, Pat Gelsinger, who is here tonight, told me they are ready to increase their investment from  \\n$20 billion to $100 billion. \\n\\nThat would be one of the biggest investments in manufacturing in American history. \\n\\nAnd all they’re waiting for is for you to pass this bill.', metadata={'source': 'state_of_the_union.txt'})]"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Here we can see it doesn't return any results because there are no relevant documents\n",
    "retriever.get_relevant_documents(\"What was the cost of training the GPT4all model?\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(a) TSNE visualization of the final training data, ten-colored\n",
      "by extracted topic.\n",
      "(b) Zoomed in view of Figure 2a. The region displayed con-\n",
      "tains generations related to personal health and wellness.\n",
      "Figure 2: The final training data was curated to ensure a diverse distribution of prompt topics and model responses.\n",
      "2.1 Reproducibility\n",
      "We release all data (including unused P3 genera-\n",
      "tions), training code, and model weights for the\n",
      "community to build upon. Please check the Git\n",
      "repository for the most up-to-date data, training\n",
      "details and checkpoints.\n",
      "2.2 Costs\n",
      "We were able to produce these models with about\n",
      "four days work, $800 in GPU costs (rented from\n",
      "Lambda Labs and Paperspace) including several\n",
      "failed trains, and $500 in OpenAI API spend.\n",
      "Our released model, gpt4all-lora, can be trained in\n",
      "about eight hours on a Lambda Labs DGX A100\n",
      "8x 80GB for a total cost of $100 .\n",
      "3 Evaluation\n",
      "We perform a preliminary evaluation of our model\n",
      "using the human evaluation data from the Self-\n",
      "Instruct paper (Wang et al., 2022). We report the\n",
      "ground truth perplexity of our model against what\n",
      "is, to our knowledge, the best openly available\n",
      "alpaca-lora model, provided by user chainyo on\n",
      "huggingface. We find that all models have very\n",
      "large perplexities on a small number of tasks, and\n",
      "report perplexities clipped to a maximum of 100.\n",
      "Models finetuned on this collected dataset ex-\n",
      "hibit much lower perplexity in the Self-Instruct\n",
      "evaluation compared to Alpaca. This evaluation is\n",
      "in no way exhaustive and further evaluation work\n",
      "Figure 3: Model Perplexities. Lower is better. Our\n",
      "models achieve stochastically lower ground truth per-\n",
      "plexities than alpaca-lora.\n",
      "remains. We welcome the reader to run the model\n",
      "locally on CPU (see Github for files) and get a\n",
      "qualitative sense of what it can do.\n",
      "4 Use Considerations\n",
      "The authors release data and training details in\n",
      "hopes that it will accelerate open LLM research,\n",
      "particularly in the domains of alignment and inter-\n",
      "pretability. GPT4All model weights and data are\n",
      "intended and licensed only for research purposes\n",
      "and any commercial use is prohibited. GPT4All\n",
      "is based on LLaMA, which has a non-commercial\n",
      "license. The assistant data is gathered from Ope-\n",
      "nAI’s GPT-3.5-Turbo, whose terms of use pro-\n"
     ]
    }
   ],
   "source": [
    "query = \"What was the cost of training the GPT4all model?\"\n",
    "results_pdf = rds_pdf.similarity_search(query)\n",
    "print(results_pdf[0].page_content)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "venv_langchain",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.2"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}