Skip to content

Commit

Permalink
Update documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
@schorndorfer
schorndorfer committed Nov 14, 2023
1 parent a57a9de commit 8e43440
Show file tree
Hide file tree
Showing 18 changed files with 401 additions and 179 deletions.
Binary file modified _images/arxiv-paper-header.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 13 additions & 14 deletions _sources/analyze-text.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,23 +11,21 @@
"cell_type": "markdown",
"metadata": {},
"source": [
" :::{admonition} Development Process\n",
" :class: important\n",
"<!-- - Come up with an idea 😀\n",
"- Identify a dataset and specify precisely what you want to extract\n",
":::{card} Development Workflow\n",
"- Identify a dataset and specify precisely what you want to do with it\n",
"- Engineer a prompt (use OpenAI Playground)\n",
"- Evaluate performance on a sample\n",
" * If performance is unacceptable, try further prompt engineering, functions, etc.\n",
" * If performance is still not good enough, try fine-tuning\n",
"- Deploy -->\n",
":::\n",
"- Deploy at scale\n",
"\n",
"```{figure} ./images/gpt-dev-cycle.png\n",
"---\n",
"width: 600px\n",
"width: 500px\n",
"name: gpt-dev-cycle\n",
"---\n",
"```"
"```\n",
":::"
]
},
{
Expand Down Expand Up @@ -101,6 +99,7 @@
"metadata": {},
"source": [
":::{admonition} Develop a Prompt\n",
":class: tip\n",
"[OpenAI playground](https://platform.openai.com/playground)\n",
":::\n",
"\n",
Expand Down Expand Up @@ -663,18 +662,18 @@
"cell_type": "markdown",
"metadata": {},
"source": [
":::{card} Structuring and Testing Code\n",
"- Structure your code into source code (functions) along with unit tests\n",
"- Functions are composed in scripts, which should have runtime tests and generate logs\n",
"- All code (source, scripts, tests) need to be in source control, typically `git`\n",
"- Save logs and output files in a secure location. Do not modify them.\n",
"```{figure} ./images/testing.png\n",
"---\n",
"width: 600px\n",
"width: 500px\n",
"name: testing\n",
"---\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": []
}
],
"metadata": {
Expand Down
78 changes: 51 additions & 27 deletions _sources/augmented-generation.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -98,15 +98,17 @@
"from langchain.vectorstores import FAISS\n",
"from langchain import OpenAI\n",
"from langchain.chains import VectorDBQAWithSourcesChain\n",
"from langchain.prompts import ChatPromptTemplate\n",
"from langchain.chat_models import ChatOpenAI\n",
"from langchain.schema.runnable import RunnablePassthrough\n",
"from langchain.schema import StrOutputParser\n",
"\n",
"\n",
"# also need\n",
"import requests\n",
"import faiss\n",
"import pickle\n",
"import numpy as np\n",
"from textwrap import wrap\n",
"from transformers import DistilBertModel, DistilBertTokenizer\n",
"import os\n"
]
},
Expand All @@ -119,7 +121,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": 3,
"metadata": {},
"outputs": [
{
Expand All @@ -128,7 +130,7 @@
"True"
]
},
"execution_count": 10,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -151,35 +153,35 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Answer: In the latest version of Python selenium, you can find an element by class name using the `find_element_by_class_name` method. Here's an example of how to use it:\n",
"Answer: In the latest version of Python Selenium, you can find an element by class name using the `find_element_by_class_name` method. Here's an example of how to use it:\n",
"\n",
"```python\n",
"from selenium import webdriver\n",
"\n",
"# Initialize the Chrome WebDriver\n",
"# Start a new web browser session\n",
"driver = webdriver.Chrome()\n",
"\n",
"# Open the webpage\n",
"driver.get(\"https://example.com\")\n",
"# Open the website\n",
"driver.get('https://www.example.com')\n",
"\n",
"# Find the element by class name\n",
"element = driver.find_element_by_class_name(\"example_class\")\n",
"element = driver.find_element_by_class_name('yourClassName')\n",
"\n",
"# Interact with the element\n",
"element.click()\n",
"# Do something with the element, for example, get its text\n",
"print(element.text)\n",
"\n",
"# Close the browser\n",
"# Close the web browser session\n",
"driver.quit()\n",
"```\n",
"\n",
"In this example, replace \"example_class\" with the actual class name of the element you want to find on the webpage.\n"
"Replace `'yourClassName'` with the actual class name of the element you're looking for.\n"
]
}
],
Expand Down Expand Up @@ -218,7 +220,7 @@
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 6,
"metadata": {},
"outputs": [
{
Expand All @@ -228,7 +230,7 @@
"<IPython.core.display.Image object>"
]
},
"execution_count": 13,
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -248,10 +250,34 @@
]
},
{
"cell_type": "markdown",
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"Created a chunk of size 4540, which is longer than the specified 1500\n",
"Created a chunk of size 1584, which is longer than the specified 1500\n",
"Created a chunk of size 52413, which is longer than the specified 1500\n",
"Created a chunk of size 1920, which is longer than the specified 1500\n",
"Created a chunk of size 3412, which is longer than the specified 1500\n",
"Created a chunk of size 3707, which is longer than the specified 1500\n",
"/Users/ambreenchaudhri/anaconda3/envs/rag/lib/python3.11/site-packages/langchain/chains/qa_with_sources/vector_db.py:65: UserWarning: `VectorDBQAWithSourcesChain` is deprecated - please use `from langchain.chains import RetrievalQAWithSourcesChain`\n",
" warnings.warn(\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Answer: In the latest version of python selenium, you can find an element by class name using the following methods: driver.findElement(By.className(\"className\")); driver.findElement(By.cssSelector(\".className\")); driver.findElementsByCssSelector(\".className\");\n",
"\n"
]
}
],
"source": [
"```python\n",
"# 1.) Load\n",
"url = \"https://www.selenium.dev/documentation/webdriver/troubleshooting/upgrade_to_selenium_4/\"\n",
"response = requests.get(url)\n",
Expand All @@ -267,7 +293,6 @@
"\n",
"# 4.) Store\n",
"store = FAISS.from_texts(chunks, embeddings, metadatas=metadata)\n",
"store.index = index\n",
"\n",
"# 5.) Retrieve\n",
"\n",
Expand All @@ -284,8 +309,7 @@
"result = chain({\"question\": question})\n",
"\n",
"# Print the answer.\n",
"print(f\"Answer: {result['answer']}\")\n",
"```"
"print(f\"Answer: {result['answer']}\")\n"
]
},
{
Expand Down Expand Up @@ -331,14 +355,14 @@
},
{
"cell_type": "code",
"execution_count": 85,
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Answer: I'm sorry for any misunderstanding, but as of current knowledge and recorded history, no elephant has ever been sent or landed on the moon. The first and only living beings to land on the moon were astronauts from NASA's Apollo missions, with the first being Apollo 11 on July 20, 1969, which had Neil Armstrong and Buzz Aldrin as the first humans to walk on the lunar surface. Sending large animals like elephants to space has numerous logistical, ethical, and wellbeing concerns.\n"
"Answer: I'm sorry for any confusion, but there actually hasn't been an elephant that has landed on the moon. The only living creatures to have made the journey to the moon are a select group of astronauts during the Apollo missions of the 1960s and 1970s conducted by NASA. One of the most noted being Neil Armstrong, the first man to walk on the moon during the Apollo 11 mission in 1969.\n"
]
}
],
Expand Down Expand Up @@ -377,7 +401,7 @@
},
{
"cell_type": "code",
"execution_count": 188,
"execution_count": 14,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -402,7 +426,7 @@
},
{
"cell_type": "code",
"execution_count": 189,
"execution_count": 15,
"metadata": {},
"outputs": [
{
Expand All @@ -411,7 +435,7 @@
"'Based on the given context, there has never been an elephant on the moon.'"
]
},
"execution_count": 189,
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
Expand Down
83 changes: 51 additions & 32 deletions analyze-text.html
View file

Large diffs are not rendered by default.

Loading

0 comments on commit 8e43440

Please sign in to comment.