From 80ed588733c42d93282c19701ffcc279ebc2ed55 Mon Sep 17 00:00:00 2001 From: Hubert Yuan Date: Thu, 30 Nov 2023 19:34:05 -0500 Subject: [PATCH] docs[patch]: Update metaphor_search.ipynb (#14093) - **Description:** Touch up of the documentation page for Metaphor Search Tool integration. Removes documentation for old built-in tool wrapper. --------- Co-authored-by: Erick Friis --- .../integrations/tools/metaphor_search.ipynb | 534 +++++++++--------- 1 file changed, 281 insertions(+), 253 deletions(-) diff --git a/docs/docs/integrations/tools/metaphor_search.ipynb b/docs/docs/integrations/tools/metaphor_search.ipynb index 1564f90d08c1b..f52ea221b2d61 100644 --- a/docs/docs/integrations/tools/metaphor_search.ipynb +++ b/docs/docs/integrations/tools/metaphor_search.ipynb @@ -2,28 +2,34 @@ "cells": [ { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "id": "4x4kQ0VcodAC" + }, "source": [ "# Metaphor Search" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "id": "V1x8wEUhodAH" + }, "source": [ - "Metaphor is a search engine fully designed to be used by LLMs. You can search and then get the contents for any page.\n", + "Metaphor is a search engine fully designed for use by LLMs. Search for documents on the internet using **natural language queries**, then retrieve **cleaned HTML content** from desired documents.\n", "\n", - "This notebook goes over how to use Metaphor search.\n", + "Unlike keyword-based search (Google), Metaphor's neural search capabilities allow it to semantically understand queries and return relevant documents. For example, we could search `\"fascinating article about cats\"` and compare the search results from [Google](https://www.google.com/search?q=fascinating+article+about+cats) and [Metaphor](https://metaphor.systems/search?q=fascinating%20article%20about%20cats&autopromptString=Here%20is%20a%20fascinating%20article%20about%20cats%3A). Google gives us SEO-optimized listicles based on the keyword \"fascinating\". Metaphor just works.\n", "\n", - "First, you need to set up the proper API keys and environment variables. Get 1000 free searches/month [here](https://platform.metaphor.systems/).\n", + "This notebook goes over how to use Metaphor Search with LangChain.\n", "\n", - "Then enter your API key as an environment variable." + "First, get a Metaphor API key and add it as an environment variable. Get 1000 free searches/month by [signing up here](https://platform.metaphor.systems/)." ] }, { "cell_type": "code", "execution_count": 1, - "metadata": {}, + "metadata": { + "id": "7GON_jdvodAI" + }, "outputs": [], "source": [ "import os\n", @@ -33,17 +39,26 @@ }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "id": "ip5_D9MkodAK" + }, "source": [ - "## Using their SDK\n", + "## Using the Metaphor SDK as LangChain Agent Tools\n", + "\n", + "The [Metaphor SDK](https://docs.metaphor.systems/) creates a client that can use the Metaphor API to perform three functions:\n", + "- `search`: Given a natural language search query, retrieve a list of search results.\n", + "- `find_similar`: Given a URL, retrieve a list of search results corresponding to webpages which are similar to the document at the provided URL.\n", + "- `get_content`: Given a list of document ids fetched from `search` or `find_similar`, get cleaned HTML content for each document.\n", "\n", - "This is the newer and more supported way to use the Metaphor API - via their SDK" + "We can use the `@tool` decorator and docstrings to create LangChain Tool wrappers that tell an LLM agent how to use Metaphor." ] }, { "cell_type": "code", - "execution_count": 2, - "metadata": {}, + "execution_count": 3, + "metadata": { + "id": "0CsJFtuFodAK" + }, "outputs": [], "source": [ "# !pip install metaphor-python" @@ -51,114 +66,87 @@ }, { "cell_type": "code", - "execution_count": 3, - "metadata": {}, + "execution_count": 4, + "metadata": { + "id": "XatXTApdodAM" + }, "outputs": [], "source": [ + "from langchain.agents import tool\n", "from metaphor_python import Metaphor\n", "\n", - "client = Metaphor(api_key=os.environ[\"METAPHOR_API_KEY\"])" - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [], - "source": [ - "from typing import List\n", + "metaphor = Metaphor(api_key=os.environ[\"METAPHOR_API_KEY\"])\n", + "\n", "\n", - "from langchain.agents import tool" - ] - }, - { - "cell_type": "code", - "execution_count": 5, - "metadata": {}, - "outputs": [], - "source": [ "@tool\n", "def search(query: str):\n", - " \"\"\"Call search engine with a query.\"\"\"\n", - " return client.search(query, use_autoprompt=True, num_results=5)\n", + " \"\"\"Search for a webpage based on the query.\"\"\"\n", + " return metaphor.search(f\"{query}\", use_autoprompt=True, num_results=5)\n", "\n", "\n", "@tool\n", - "def get_contents(ids: List[str]):\n", - " \"\"\"Get contents of a webpage.\n", - "\n", - " The ids passed in should be a list of ids as fetched from `search`.\n", + "def find_similar(url: str):\n", + " \"\"\"Search for webpages similar to a given URL.\n", + " The url passed in should be a URL returned from `search`.\n", " \"\"\"\n", - " return client.get_contents(ids)\n", + " return metaphor.find_similar(url, num_results=5)\n", "\n", "\n", "@tool\n", - "def find_similar(url: str):\n", - " \"\"\"Get search results similar to a given URL.\n", - "\n", - " The url passed in should be a URL returned from `search`\n", + "def get_contents(ids: list[str]):\n", + " \"\"\"Get the contents of a webpage.\n", + " The ids passed in should be a list of ids returned from `search`.\n", " \"\"\"\n", - " return client.find_similar(url, num_results=5)" - ] - }, - { - "cell_type": "code", - "execution_count": 6, - "metadata": {}, - "outputs": [], - "source": [ + " return metaphor.get_contents(ids)\n", + "\n", + "\n", "tools = [search, get_contents, find_similar]" ] }, { "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Use in an agent" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [], + "metadata": { + "id": "sVe2ca9OodAO" + }, "source": [ - "from langchain.chat_models import ChatOpenAI\n", + "### Providing Metaphor Tools to an Agent\n", "\n", - "llm = ChatOpenAI(temperature=0)" + "We can provide the Metaphor tools we just created to a LangChain `OpenAIFunctionsAgent`. When asked to `Summarize for me a fascinating article about cats`, the agent uses the `search` tool to perform a Metaphor search with an appropriate search query, uses the `get_contents` tool to perform Metaphor content retrieval, and then returns a summary of the retrieved content." ] }, { "cell_type": "code", - "execution_count": 18, - "metadata": {}, + "execution_count": 5, + "metadata": { + "id": "x72pY95WodAP" + }, "outputs": [], "source": [ - "from langchain.agents import OpenAIFunctionsAgent\n", + "from langchain.agents import AgentExecutor, OpenAIFunctionsAgent\n", + "from langchain.chat_models import ChatOpenAI\n", "from langchain.schema import SystemMessage\n", "\n", + "llm = ChatOpenAI(temperature=0)\n", + "\n", "system_message = SystemMessage(\n", - " content=\"You are a web researcher who uses search engines to look up information.\"\n", + " content=\"You are a web researcher who answers user questions by looking up information on the internet and retrieving contents of helpful documents. Cite your sources.\"\n", ")\n", - "prompt = OpenAIFunctionsAgent.create_prompt(system_message=system_message)\n", - "agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=prompt)" - ] - }, - { - "cell_type": "code", - "execution_count": 19, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain.agents import AgentExecutor\n", "\n", + "agent_prompt = OpenAIFunctionsAgent.create_prompt(system_message)\n", + "agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=agent_prompt)\n", "agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)" ] }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 6, "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "jTku-3OmodAP", + "outputId": "d587ed66-3b82-4bc8-e8cd-f5278e0c14a6", "scrolled": false }, "outputs": [ @@ -170,242 +158,282 @@ "\n", "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", "\u001b[32;1m\u001b[1;3m\n", - "Invoking: `search` with `{'query': 'hottest AI agent startups'}`\n", - "\n", - "\n", - "\u001b[0m\u001b[36;1m\u001b[1;3mSearchResponse(results=[Result(title='A Search Engine for Machine Intelligence', url='https://bellow.ai/', id='bdYc6hvHww_JvLv9k8NhPA', score=0.19460266828536987, published_date='2023-01-01', author=None, extract=None), Result(title='Adept: Useful General Intelligence', url='https://www.adept.ai/', id='aNBppxBZvQRZMov6sFVj9g', score=0.19103890657424927, published_date='2000-01-01', author=None, extract=None), Result(title='HiOperator | Generative AI-Enhanced Customer Service', url='https://www.hioperator.com/', id='jieb6sB53mId3EDo0z-SDw', score=0.18549954891204834, published_date='2000-01-01', author=None, extract=None), Result(title='Home - Stylo', url='https://www.askstylo.com/', id='kUiCuCjJYMD4N0NXdCtqlQ', score=0.1837376356124878, published_date='2000-01-01', author=None, extract=None), Result(title='DirectAI', url='https://directai.io/?utm_source=twitter&utm_medium=raw_message&utm_campaign=first_launch', id='45iSS8KnJ9tL1ilPg3dL9A', score=0.1835256814956665, published_date='2023-01-01', author=None, extract=None), Result(title='Sidekick AI | Customer Service Automated', url='https://www.sidekickai.co/', id='nCoPMUtqWQqhUvsdTjJT6A', score=0.18215584754943848, published_date='2020-01-01', author=None, extract=None), Result(title='Hebbia - Search, Reinvented', url='https://www.hebbia.ai/', id='Zy0YaekZdd4rurPQKkys7A', score=0.1799020767211914, published_date='2023-01-01', author=None, extract=None), Result(title='AI.XYZ', url='https://www.ai.xyz/', id='A5c1ePEvsaQeml2Kui_-vA', score=0.1797989457845688, published_date='2023-01-01', author=None, extract=None), Result(title='Halist AI', url='https://halist.ai/', id='-lKPLSb4N4dgMZlTgoDvJg', score=0.17975398898124695, published_date='2023-03-01', author=None, extract=None), Result(title='Clone your best expert', url='https://airin.ai/', id='_XIjx1YLPfI4cKePIEc_bQ', score=0.17957791686058044, published_date='2016-02-12', author=None, extract=None)], api=)\u001b[0m\u001b[32;1m\u001b[1;3m\n", - "Invoking: `get_contents` with `{'ids': ['bdYc6hvHww_JvLv9k8NhPA', 'aNBppxBZvQRZMov6sFVj9g', 'jieb6sB53mId3EDo0z-SDw', 'kUiCuCjJYMD4N0NXdCtqlQ', '45iSS8KnJ9tL1ilPg3dL9A', 'nCoPMUtqWQqhUvsdTjJT6A', 'Zy0YaekZdd4rurPQKkys7A', 'A5c1ePEvsaQeml2Kui_-vA', '-lKPLSb4N4dgMZlTgoDvJg', '_XIjx1YLPfI4cKePIEc_bQ']}`\n", - "\n", - "\n", - "\u001b[0m\u001b[33;1m\u001b[1;3mGetContentsResponse(contents=[DocumentContent(id='bdYc6hvHww_JvLv9k8NhPA', url='https://bellow.ai/', title='A Search Engine for Machine Intelligence', extract=\"

More Opinions

Get responses from multiple AIs

Don't rely on a single source of truth, explore the full space of machine intelligence and get highly tailored results.

\"), DocumentContent(id='aNBppxBZvQRZMov6sFVj9g', url='https://www.adept.ai/', title='Adept: Useful General Intelligence', extract='

Useful
General
Intelligence

'), DocumentContent(id='jieb6sB53mId3EDo0z-SDw', url='https://www.hioperator.com/', title='HiOperator | Generative AI-Enhanced Customer Service', extract=\"

Generative AI-Enhanced Customer Support Automation

Flexible, Scalable Customer Support

Why HiOperator?

Truly scalable customer service

A digital-first customer service provider that changes all the rules of what's possible. Scalable. 100% US-Based. Effortless. HiOperator is the digital payoff.

Next-Gen Customer Service

Scaling with HiOperator's Superagents

HiOperator is only possible in the digital era. Our revolutionary software connects with your systems to empower our agents to learn quickly and deliver incredible accuracy.

Train Us Once

We handle all of the recruiting, hiring, and training moving forward. Never have to deal with another classroom retraining or head count headaches.

Send Us Tickets

We pull tickets automatically from your preferred CRM vendor into our custom system. You have full control over how and when we get tickets.

Pay per resolution

We charge for each conversation we solve. No onboarding fees. No hourly rates. Pay for what you use.

Customer Experience

Insights &Â\\xa0News

Let's transform your customer service.

We can onboard in a matter of days and we offer highly flexible contracts. Whether you need a large team to handle your support or some overflow assistance, getting started is easy.

We can onboard in a matter of days and we offer highly flexible contracts. Whether you need a large team to handle your support or some overflow assistance, getting started is easy.

\"), DocumentContent(id='kUiCuCjJYMD4N0NXdCtqlQ', url='https://www.askstylo.com/', title='Home - Stylo', extract='

Stop angry customers from breaking support

â\\x80\\x9cWe solve 99 tickets perfectly ð\\x9f\\x98\\x87 but the 1 we miss lands in the CEOâ\\x80\\x99s inbox ð\\x9f\\x98«â\\x80\\x9d

â\\x80\\x8dThat 1 costly ticket breaks your process, metrics, and the will of your team. Angry customers make support teams less effective, which makes customers angrier in return.
â\\x80\\x8d

Stylo is AI that tells you where to most effectively spend your time to improve the customer experience. This leads to happier customers, employees, and reduces churn.

â\\x80\\x8dNo setup, no learning curve, just plug it in and go.

â\\x80\\x9cIâ\\x80\\x99m able to better manage the team because I can pinpoint gaps in the teamâ\\x80\\x99s knowledge or training, and find room for process improvements.â\\x80\\x9d

'), DocumentContent(id='45iSS8KnJ9tL1ilPg3dL9A', url='https://directai.io/?utm_source=twitter&utm_medium=raw_message&utm_campaign=first_launch', title='DirectAI', extract=\"

Vision models without training data.

Build and deploy powerful computer vision models with plain language.
No code or training required.

Fundamentally different.

We use large language models and zero-shot learning to instantly build models that fit your description.


We're removing the last major barrier to creating custom models -
training data.

Deploy and iterate in seconds with DirectAI

• Don't spend time assembling training data.
• Don't pay a third party to label your data.
• Don't pay to train your model.
• Don't spend months finetuning your model's behavior.

Venture-backed.

Based in NYC.

We're changing how people use AI in the real world.

Come talk to us on .

\"), DocumentContent(id='nCoPMUtqWQqhUvsdTjJT6A', url='https://www.sidekickai.co/', title='Sidekick AI | Customer Service Automated', extract='

Hi, I am an AI named Jenny, working at Pizza Planet. How can I help you today?

How much are large pizzas with 1 topping?

For most toppings, a large with one topping would be $10.99.

Ok, can I order a large with pepperoni

Sure! Takeout or delivery?

Alright, order placed. See you at 5 pm!

Meet Sidekick

\\n Sidekick is an AI agent built to hold natural and dynamic conversations with your customers and talk just like a human.

Built on the world\\'s most advanced AI models, Sidekick pushes the state of the art in natural conversation and converses seamlessly with your customers.\\n

Try it out ➜

Try it out ↓

An AI agent designed for service-led growth.

Personal

Every customer is different, and has unique needs. Our agents are built to provide personalized service depending on the customer\\'s needs.

Fast

Unlike humans, our Sidekicks respond near-instantly, any time of the day. Your customers won\\'t wait for service ever again.

Effective

Customers love great service, and Sidekick delivers. Grow revenue by solving issues in minutes instead of hours, and providing personalized support to each customer.

Integrating with your tools.

Wherever your customers are.

\\n Sidekick takes an omnichannel approach to customer service, aggregating all customer interactions across all platforms in one area. Currently most social media platforms are supported, along with website embeddings and API integration.\\n

On the web.

Sidekick makes adding a live chat to your website as simple as copy and pasting a single line of code.

Chat bubbles discretely sit in the bottom right corner and provide a smooth conversation experience, with AI and human agents alike.

On Facebook.

Sidekick integrates with your Facebook pages to make live customer service one click away.

Customers can reach your agent and get service without ever leaving Messenger.

On Instagram.

E-Commerce on Instagram is especially demanding for customer service.

Sidekick integrates easily with Instagram accounts to put a live agent one click away.

On Twitter.

Customers are spending more time on Twitter, which means businesses should provide customer service right on the platform.

Sidekick integrates easily with Twitter accounts to put a live agent one click away.

Anywhere you want.

Our API provides programmatic access to your Sidekick agent to integrate into your own app.

We\\'ve built simple abstractions over the chat interface to make it easy to work with our API.

Endpoints

POST

https://www.api.sidekickai.co/converse

Sample Request

{\\n \"access_token\": \"KjZUZBWAOKwgLWAlVFyL\",\\n \"conversation_id\": \"23874\",\\n \"body\": \"How much is a large 2 topping?\"\\n}

Sample Response

{\\n \"response\": \"A large'), DocumentContent(id='Zy0YaekZdd4rurPQKkys7A', url='https://www.hebbia.ai/', title='Hebbia - Search, Reinvented', extract=\"

Direct to the point
with cutting-edge AI.

Stop relying on archaic software, traditional Q&A emails, or waiting for deal partners. Get answers on your own time with accuracy that you can't replicate with humans.
â\\x80\\x8d

HebbiaÂ\\xa0retrieves every answer, even insights humans overlook.

\"), DocumentContent(id='A5c1ePEvsaQeml2Kui_-vA', url='https://www.ai.xyz/', title='AI.XYZ', extract='
\\n \\n \\n
\\n \\n \\n \\n \\n \\n
\\n

Go be human

\\n

\\n

Let your AI deal with the rest

\\n

Design your own AI with AI.XYZ

\\n \\n \\n \\n

\\n \\n
\\n \\n \\n \\n \\n

\\n

The digital world was designed to make us more productive but now navigating it all has become its own job.

\\n

\\n \\n \\n \\n \\n
\\n
\\n \\n \\n \\n \\n \\n \\n \\n \\n

\\n \\n \\n
\\n

\\n

Take life a little easier

\\n

\\n \\n \\n \\n

\\n \\n

\\n

Tackles info
overload

\\n

\\n

“Like ChatGPT, but way more proactive and useful because it’s designed by me, for only me”

\\n

\\n \\n \\n \\n

\\n \\n

\\n

Never sits
around

\\n

\\n

“Even if I’m not interacting with it, my AI looks for ways to simplify my day, surprising me with useful ideas”

\\n

\\n \\n \\n \\n

\\n \\n

\\n

Supports and
inspires

\\n

\\n

“It takes things off my plate, but also cheers me on throughout the day — helping me navigate it all”

\\n

\\n \\n \\n
\\n \\n \\n \\n \\n

\\n

Create your AI in 3 simple steps:

\\n

\\n

STEP ONE

Pick a face and voice

Choose from our library of characters or add your own unique face and voice.

\\n
\\n \\n \\n \\n

\\n \\n
\\n

STEP TWO

Create your AI’s persona and memory

Decide who your AI is, its purpose and what it will help you with. Paste information that you want your AI to know.

\\n
\\n \\n \\n \\n

\\n \\n
\\n

STEP THREE

Get started

Ask your AI to help you with ideas and support throughout your day. Eventually it will be able to proactively support you.

\\n
\\n \\n \\n \\n

\\n \\n
\\n \\n \\n \\n \\n
\\n
\\n \\n \\n \\n \\n \\n \\n \\n \\n

\\n \\n \\n
\\n

\\n

Start training your AI to do things for you

\\n

\\n \\n \\n
\\n \\n
\\n \\n \\n \\n \\n \\n

For $15,000, you can get your pet a new kidney.

By Sarah Zhang

Photographs by Caroline Tompkins for The Atlantic

Sherlock donated a kidney in 2019. (Caroline Tompkins for The Atlantic)

This article was featured in One Story to Read Today, a newsletter in which our editors recommend a single must-read from The Atlantic, Monday through Friday. Sign up for it here.      

When I first met Strawberry, age 16, she was lying on her back, paws akimbo. Her cat belly was shaved bare, and black stitches ran several inches down her naked pink skin.

A radiologist squirted ultrasound goop on her abdomen while two veterinary students in dark-blue scrubs gently held down her legs—not that this was really necessary. Strawberry was too tired, too drugged, or simply too out of it from her surgery the previous day to protest. In the dim light of the radiology room, her pupils were dilated into deep black pools. She slowly turned her head toward me. She turned away. She looked around at the small crowd of doctors and students surrounding her, as if to wonder what on God’s green earth had happened for her to end up like this.

Explore the December 2022 Issue

Check out more from this issue and find your next story to read.

View More

What had happened was that Strawberry had received a kidney transplant. A surgical team at the University of Georgia had shaved off patches of her long ginger fur, inserting catheters in her leg and neck to deliver the cocktail of drugs she would need during her hospital stay: anesthesia, painkillers, antibiotics, blood thinners, and immunosuppressants. Then a surgeon named Chad Schmiedt carefully cut down the midline of her belly—past the two shriveled kidneys that were no longer doing their job and almost to her groin. Next, he stitched into place a healthy new kidney, freshly retrieved from a living donor just hours earlier.

Schmiedt is one of only a few surgeons who perform transplants on cats, and is therefore one of the world’s foremost experts at connecting cat kidneys. When he first greeted me with a broad smile and a handshake, I was struck by how his large, callused hand engulfed mine. In the operating room, though, his hands work with microscopic precision, stitching up arteries and veins only millimeters wide. This is the hardest part, he told me, like sewing “wet rice paper.” Once the donor kidney was in place, it flushed pink and Schmiedt closed Strawberry back up. (As in human transplants, the old kidneys can stay in place.) It was then a matter of waiting for her to wake up and pee. She had done both by the time of her ultrasound.

Not that Strawberry could understand any of this—or that any cat understands why we humans insist on bringing them to vet offices to be poked and prodded by strangers. But without the transplant, she would die of kidney failure, an affliction akin to being gradually poisoned from within. Other treatments could slow her kidney disease, which is common in older cats, but they could not stop it. This is why Strawberry’s owner decided to spend $15,000 on a kidney—a last resort to save her life, or at least extend it.

I didn’t meet her owner in the hospital that day. Strawberry would need to be hospitalized for at least a week afterAuthor: None\u001b[0m\u001b[32;1m\u001b[1;3mHere is a fascinating article about cats titled \"How Much Would You Pay to Save Your Cat’s Life?\" by Sarah Zhang. The article discusses the cost of critical care for pets and the lengths that some owners are willing to go to save their cat's life. It highlights a case of a cat named Strawberry who underwent a kidney transplant, a procedure performed by one of the few surgeons who specialize in cat transplants. The article explores the emotional and financial decisions involved in providing life-saving treatments for pets. You can read the full article [here](https://www.theatlantic.com/magazine/archive/2022/12/paying-for-pet-critical-care-cost-health-insurance/671896/).\u001b[0m\n", "\n", "\u001b[1m> Finished chain.\u001b[0m\n" ] }, { "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, "text/plain": [ - "\"Here are some of the hottest AI agent startups and what they do:\\n\\n1. [Bellow AI](https://bellow.ai/): This startup provides a search engine for machine intelligence. It allows users to get responses from multiple AIs, exploring the full space of machine intelligence and getting highly tailored results.\\n\\n2. [Adept AI](https://www.adept.ai/): Adept is focused on creating useful general intelligence.\\n\\n3. [HiOperator](https://www.hioperator.com/): HiOperator offers generative AI-enhanced customer support automation. It provides scalable, digital-first customer service and uses its software to empower agents to learn quickly and deliver accurate results.\\n\\n4. [Stylo](https://www.askstylo.com/): Stylo uses AI to help manage customer support, identifying where to most effectively spend time to improve the customer experience.\\n\\n5. [DirectAI](https://directai.io/): DirectAI allows users to build and deploy powerful computer vision models with plain language, without the need for code or training.\\n\\n6. [Sidekick AI](https://www.sidekickai.co/): Sidekick AI is built to hold natural and dynamic conversations with customers, providing personalized service depending on the customer's needs.\\n\\n7. [Hebbia](https://www.hebbia.ai/): Hebbia is reinventing search with cutting-edge AI, retrieving every answer, even insights humans overlook.\\n\\n8. [AI.XYZ](https://www.ai.xyz/): AI.XYZ allows users to design their own AI, tackling information overload and providing support and inspiration throughout the day.\\n\\n9. [Halist AI](https://halist.ai/): Halist AI provides optimized access to ChatGPT, powered by OpenAI GPT-3 and GPT-4, on mobile.\\n\\n10. [Airin](https://airin.ai/): Airin clones how your top expert solves problems in as little as 2 hours, creating an AI companion for the rest of your team. It automates remote coaching for new hires and dramatically reduces time to productivity.\\n\"" + "'Here is a fascinating article about cats titled \"How Much Would You Pay to Save Your Cat’s Life?\" by Sarah Zhang. The article discusses the cost of critical care for pets and the lengths that some owners are willing to go to save their cat\\'s life. It highlights a case of a cat named Strawberry who underwent a kidney transplant, a procedure performed by one of the few surgeons who specialize in cat transplants. The article explores the emotional and financial decisions involved in providing life-saving treatments for pets. You can read the full article [here](https://www.theatlantic.com/magazine/archive/2022/12/paying-for-pet-critical-care-cost-health-insurance/671896/).'" ] }, - "execution_count": 20, + "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ - "agent_executor.run(\"Find the hottest AI agent startups and what they do\")" + "agent_executor.run(\"Summarize for me a fascinating article about cats.\")" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "id": "e3FHjxT-RoIH" + }, "source": [ - "## Using the tool wrapper\n", + "## Advanced Metaphor Features\n", "\n", - "This is the old way of using Metaphor - through our own in-house integration." - ] - }, - { - "cell_type": "code", - "execution_count": 2, - "metadata": {}, - "outputs": [], - "source": [ - "from langchain.utilities import MetaphorSearchAPIWrapper" + "Metaphor supports powerful filters by domain and date. We can provide a more powerful `search` tool to the agent that lets it decide to apply filters if they are useful for the objective. See all of Metaphor's search features [here](https://github.com/metaphorsystems/metaphor-python/)." ] }, { "cell_type": "code", - "execution_count": 3, - "metadata": {}, + "execution_count": 7, + "metadata": { + "id": "fFPl4dUs7F_v" + }, "outputs": [], "source": [ - "search = MetaphorSearchAPIWrapper()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Call the API\n", - "`results` takes in a Metaphor-optimized search query and a number of results (up to 500). It returns a list of results with title, url, author, and creation date." - ] - }, - { - "cell_type": "code", - "execution_count": 4, - "metadata": {}, - "outputs": [ - { - "data": { - "text/plain": [ - "[{'title': 'Core Views on AI Safety: When, Why, What, and How',\n", - " 'url': 'https://www.anthropic.com/index/core-views-on-ai-safety',\n", - " 'author': None,\n", - " 'published_date': '2023-03-08'},\n", - " {'title': 'Extinction Risk from Artificial Intelligence',\n", - " 'url': 'https://aisafety.wordpress.com/',\n", - " 'author': None,\n", - " 'published_date': '2013-10-08'},\n", - " {'title': 'The simple picture on AI safety - LessWrong',\n", - " 'url': 'https://www.lesswrong.com/posts/WhNxG4r774bK32GcH/the-simple-picture-on-ai-safety',\n", - " 'author': 'Alex Flint',\n", - " 'published_date': '2018-05-27'},\n", - " {'title': 'No Time Like The Present For AI Safety Work',\n", - " 'url': 'https://slatestarcodex.com/2015/05/29/no-time-like-the-present-for-ai-safety-work/',\n", - " 'author': None,\n", - " 'published_date': '2015-05-29'},\n", - " {'title': 'A plea for solutionism on AI safety - LessWrong',\n", - " 'url': 'https://www.lesswrong.com/posts/ASMX9ss3J5G3GZdok/a-plea-for-solutionism-on-ai-safety',\n", - " 'author': 'Jasoncrawford',\n", - " 'published_date': '2023-06-09'},\n", - " {'title': 'The Artificial Intelligence Revolution: Part 1 - Wait But Why',\n", - " 'url': 'https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html',\n", - " 'author': 'Tim Urban',\n", - " 'published_date': '2015-01-22'},\n", - " {'title': 'Anthropic: Core Views on AI Safety: When, Why, What, and How - EA Forum',\n", - " 'url': 'https://forum.effectivealtruism.org/posts/uGDCaPFaPkuxAowmH/anthropic-core-views-on-ai-safety-when-why-what-and-how',\n", - " 'author': 'Jonmenaster',\n", - " 'published_date': '2023-03-09'},\n", - " {'title': \"[Linkpost] Sam Altman's 2015 Blog Posts Machine Intelligence Parts 1 & 2 - LessWrong\",\n", - " 'url': 'https://www.lesswrong.com/posts/QnBZkNJNbJK9k5Xi7/linkpost-sam-altman-s-2015-blog-posts-machine-intelligence',\n", - " 'author': 'Olivia Jimenez',\n", - " 'published_date': '2023-04-28'},\n", - " {'title': 'The Proof of Doom - LessWrong',\n", - " 'url': 'https://www.lesswrong.com/posts/xBrpph9knzWdtMWeQ/the-proof-of-doom',\n", - " 'author': 'Johnlawrenceaspden',\n", - " 'published_date': '2022-03-09'},\n", - " {'title': \"Anthropic's Core Views on AI Safety - LessWrong\",\n", - " 'url': 'https://www.lesswrong.com/posts/xhKr5KtvdJRssMeJ3/anthropic-s-core-views-on-ai-safety',\n", - " 'author': 'Zac Hatfield-Dodds',\n", - " 'published_date': '2023-03-09'}]" - ] - }, - "execution_count": 4, - "metadata": {}, - "output_type": "execute_result" - } - ], - "source": [ - "search.results(\"The best blog post about AI safety is definitely this: \", 10)" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "### Adding filters\n", - "We can also add filters to our search. \n", + "from langchain.agents import tool\n", + "from metaphor_python import Metaphor\n", "\n", - "include_domains: Optional[List[str]] - List of domains to include in the search. If specified, results will only come from these domains. Only one of include_domains and exclude_domains should be specified.\n", + "metaphor = Metaphor(api_key=os.environ[\"METAPHOR_API_KEY\"])\n", "\n", - "exclude_domains: Optional[List[str]] - List of domains to exclude in the search. If specified, results will only come from these domains. Only one of include_domains and exclude_domains should be specified.\n", "\n", - "start_crawl_date: Optional[str] - \"Crawl date\" refers to the date that Metaphor discovered a link, which is more granular and can be more useful than published date. If start_crawl_date is specified, results will only include links that were crawled after start_crawl_date. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ)\n", + "@tool\n", + "def search(query: str, include_domains=None, start_published_date=None):\n", + " \"\"\"Search for a webpage based on the query.\n", + " Set the optional include_domains (list[str]) parameter to restrict the search to a list of domains.\n", + " Set the optional start_published_date (str) parameter to restrict the search to documents published after the date (YYYY-MM-DD).\n", + " \"\"\"\n", + " return metaphor.search(\n", + " f\"{query}\",\n", + " use_autoprompt=True,\n", + " num_results=5,\n", + " include_domains=include_domains,\n", + " start_published_date=start_published_date,\n", + " )\n", "\n", - "end_crawl_date: Optional[str] - \"Crawl date\" refers to the date that Metaphor discovered a link, which is more granular and can be more useful than published date. If endCrawlDate is specified, results will only include links that were crawled before end_crawl_date. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ)\n", "\n", - "start_published_date: Optional[str] - If specified, only links with a published date after start_published_date will be returned. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ). Note that for some links, we have no published date, and these links will be excluded from the results if start_published_date is specified.\n", + "@tool\n", + "def find_similar(url: str):\n", + " \"\"\"Search for webpages similar to a given URL.\n", + " The url passed in should be a URL returned from `search`.\n", + " \"\"\"\n", + " return metaphor.find_similar(url, num_results=5)\n", "\n", - "end_published_date: Optional[str] - If specified, only links with a published date before end_published_date will be returned. Must be specified in ISO 8601 format (YYYY-MM-DDTHH:MM:SSZ). Note that for some links, we have no published date, and these links will be excluded from the results if end_published_date is specified.\n", "\n", - "See full docs [here](https://metaphorapi.readme.io/)." - ] - }, - { - "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], - "source": [ - "search.results(\n", - " \"The best blog post about AI safety is definitely this: \",\n", - " 10,\n", - " include_domains=[\"lesswrong.com\"],\n", - " start_published_date=\"2019-01-01\",\n", - ")" + "@tool\n", + "def get_contents(ids: list[str]):\n", + " \"\"\"Get the contents of a webpage.\n", + " The ids passed in should be a list of ids returned from `search`.\n", + " \"\"\"\n", + " return metaphor.get_contents(ids)\n", + "\n", + "\n", + "tools = [search, get_contents, find_similar]" ] }, { "cell_type": "markdown", - "metadata": {}, + "metadata": { + "id": "BZlScmg1b5Km" + }, "source": [ - "### Use Metaphor as a tool\n", - "Metaphor can be used as a tool that gets URLs that other tools such as browsing tools." + "Now we ask the agent to summarize an article with constraints on domain and publish date. We will use a GPT-4 agent for extra powerful reasoning capability to support more complex tool usage.\n", + "\n", + "The agent correctly uses the search filters to find an article with the desired constraints, and once again retrieves the content and returns a summary." ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, + "execution_count": 8, + "metadata": { + "id": "7ns7V9FNTyZM" + }, "outputs": [], "source": [ - "%pip install playwright\n", - "from langchain.agents.agent_toolkits import PlayWrightBrowserToolkit\n", - "from langchain.tools.playwright.utils import (\n", - " create_async_playwright_browser, # A synchronous browser is available, though it isn't compatible with jupyter.\n", - ")\n", + "from langchain.agents import AgentExecutor, OpenAIFunctionsAgent\n", + "from langchain.chat_models import ChatOpenAI\n", + "from langchain.schema import SystemMessage\n", "\n", - "async_browser = create_async_playwright_browser()\n", - "toolkit = PlayWrightBrowserToolkit.from_browser(async_browser=async_browser)\n", - "tools = toolkit.get_tools()\n", + "llm = ChatOpenAI(temperature=0, model=\"gpt-4\")\n", "\n", - "tools_by_name = {tool.name: tool for tool in tools}\n", - "print(tools_by_name.keys())\n", - "navigate_tool = tools_by_name[\"navigate_browser\"]\n", - "extract_text = tools_by_name[\"extract_text\"]" + "system_message = SystemMessage(\n", + " content=\"You are a web researcher who answers user questions by looking up information on the internet and retrieving contents of helpful documents. Cite your sources.\"\n", + ")\n", + "\n", + "agent_prompt = OpenAIFunctionsAgent.create_prompt(system_message)\n", + "agent = OpenAIFunctionsAgent(llm=llm, tools=tools, prompt=agent_prompt)\n", + "agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)" ] }, { "cell_type": "code", - "execution_count": null, - "metadata": {}, - "outputs": [], + "execution_count": 9, + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 1000 + }, + "id": "fmx2GsaIT0eV", + "outputId": "e0689853-b9b7-4010-ab1d-05aca71f7f5e" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "\n", + "\n", + "\u001b[1m> Entering new AgentExecutor chain...\u001b[0m\n", + "\u001b[32;1m\u001b[1;3m\n", + "Invoking: `search` with `{'query': 'AI article', 'include_domains': ['lesswrong.com'], 'start_published_date': '2023-10-01'}`\n", + "\n", + "\n", + "\u001b[0m\u001b[36;1m\u001b[1;3mTitle: A dialectical view of the history of AI, Part 1: We’re only in the antithesis phase. [A synthesis is in the future.]\n", + "URL: https://www.lesswrong.com/posts/K9GP2ZSsugRAGfFns/a-dialectical-view-of-the-history-of-ai-part-1-we-re-only-in\n", + "ID: 45-Cs6bdV4_HB8epTO0utg\n", + "Score: 0.14216266572475433\n", + "Published Date: 2023-11-16\n", + "Author: Bill Benzon\n", + "Extract: None\n", + "\n", + "Title: Possible OpenAI's Q* breakthrough and DeepMind's AlphaGo-type systems plus LLMs\n", + "URL: https://www.lesswrong.com/posts/JnM3EHegiBePeKkLc/possible-openai-s-q-breakthrough-and-deepmind-s-alphago-type\n", + "ID: BPJkDFi_bcpo7T19zlj0Ag\n", + "Score: 0.13007280230522156\n", + "Published Date: 2023-11-23\n", + "Author: Burny\n", + "Extract: None\n", + "\n", + "Title: Architects of Our Own Demise: We Should Stop Developing AI\n", + "URL: https://www.lesswrong.com/posts/bHHrdXwrCj2LRa2sW/architects-of-our-own-demise-we-should-stop-developing-ai\n", + "ID: IMLGjczHCLOuoB6w_vemkA\n", + "Score: 0.12803198397159576\n", + "Published Date: 2023-10-26\n", + "Author: Roko\n", + "Extract: None\n", + "\n", + "Title: \"Benevolent [ie, Ruler] AI is a bad idea\" and a suggested alternative\n", + "URL: https://www.lesswrong.com/posts/bwL63TrCo4RaHc3WJ/benevolent-ie-ruler-ai-is-a-bad-idea-and-a-suggested\n", + "ID: RvRnTH2cpV186d98SFbb9w\n", + "Score: 0.12487411499023438\n", + "Published Date: 2023-11-19\n", + "Author: The Gears\n", + "Extract: None\n", + "\n", + "Title: \"Benevolent [Ruler] AI is a bad idea\" and a suggested alternative\n", + "URL: https://www.lesswrong.com/posts/bwL63TrCo4RaHc3WJ/benevolent-ruler-ai-is-a-bad-idea-and-a-suggested\n", + "ID: PjwqS66am-duoo_2ueRpQw\n", + "Score: 0.12358281761407852\n", + "Published Date: 2023-11-19\n", + "Author: The Gears\n", + "Extract: None\n", + "\n", + "Autoprompt String: Here is an interesting article about AI:\u001b[0m\u001b[32;1m\u001b[1;3m\n", + "Invoking: `get_contents` with `{'ids': ['45-Cs6bdV4_HB8epTO0utg']}`\n", + "\n", + "\n", + "\u001b[0m\u001b[33;1m\u001b[1;3mID: 45-Cs6bdV4_HB8epTO0utg\n", + "URL: https://www.lesswrong.com/posts/K9GP2ZSsugRAGfFns/a-dialectical-view-of-the-history-of-ai-part-1-we-re-only-in\n", + "Title: A dialectical view of the history of AI, Part 1: We’re only in the antithesis phase. [A synthesis is in the future.]\n", + "Extract:

Cross-posted from New Savanna.

The idea that history proceeds by way of dialectical change is due primarily to Hegel and Marx. While I read bit of both early in my career, I haven’t been deeply influenced by either of them. Nonetheless I find the notion of dialectical change working out through history to be a useful way of thinking about the history of AI. Because it implies that that history is more than just one thing of another.

This dialectical process is generally schematized as a movement from a thesis, to an antithesis, and finally, to a synthesis on a “higher level,” whatever that is. The technical term is Aufhebung. Wikipedia:

In Hegel, the term Aufhebung has the apparently contradictory implications of both preserving and changing, and eventually advancement (the German verb aufheben means \"to cancel\", \"to keep\" and \"to pick up\"). The tension between these senses suits what Hegel is trying to talk about. In sublation, a term or concept is both preserved and changed through its dialectical interplay with another term or concept. Sublation is the motor by which the dialectic functions.

So, why do I think the history of AI is best conceived in this way? The first era, THESIS, running from the 1950s up through and into the 1980s, was based on top-down deductive symbolic methods. The second era, ANTITHESIS, which began its ascent in the 1990s and now reigns, is based on bottom-up statistical methods. These are conceptually and computationally quite different, opposite, if you will. As for the third era, SYNTHESIS, well, we don’t even know if there will be a third era. Perhaps the second, the current, era will take us all the way, whatever that means. Color me skeptical. I believe there will be a third era, and that it will involve a synthesis of conceptual ideas computational techniques from the previous eras.

Note, though, that I will be concentrating on efforts to model language. In the first place, that’s what I know best. More importantly, however, it is the work on language that is currently evoking the most fevered speculations about the future of AI.

Let’s take a look. Find a comfortable chair, adjust the lighting, pour yourself a drink, sit back, relax, and read. This is going to take a while.

Symbolic AI: Thesis

The pursuit of artificial intelligence started back in the 1950s it began with certain ideas and certain computational capabilities. The latter were crude and radically underpowered by today’s standards. As for the ideas, we need two more or less independent starting points. One gives us the term “artificial intelligence” (AI), which John McCarthy coined in connection with a conference held at Dartmouth in 1956. The other is associated with the pursuit of machine translation (MT) which, in the United States, meant translating Russian technical documents into English. MT was funded primarily by the Defense Department.

The goal of MT was practical, relentlessly practical. There was no talk of intelligence and Turing tests and the like. The only thing that mattered was being able to take a Russian text, feed it into a computer, and get out a competent English translation of that text. Promises was made, but little was delivered. The Defense Department pulled the plug on that work in the mid-1960s. Researchers in MT then proceeded to rebrand themselves as investigators ofAuthor: None\u001b[0m\u001b[32;1m\u001b[1;3mThe article titled \"A dialectical view of the history of AI, Part 1: We’re only in the antithesis phase. [A synthesis is in the future.]\" on LessWrong provides a unique perspective on the evolution of AI. The author, Bill Benzon, uses the concept of dialectical change, primarily attributed to Hegel and Marx, to explain the history of AI. \n", + "\n", + "The dialectical process is described as a movement from a thesis, to an antithesis, and finally, to a synthesis on a higher level. The author applies this concept to AI by identifying three eras. The first era, the THESIS, ran from the 1950s to the 1980s and was based on top-down deductive symbolic methods. The second era, the ANTITHESIS, began in the 1990s and is based on bottom-up statistical methods. These two eras are conceptually and computationally quite different. \n", + "\n", + "As for the third era, the SYNTHESIS, the author speculates that it will involve a synthesis of conceptual ideas and computational techniques from the previous eras. However, it is unclear if this era will occur or if the current era will take us \"all the way\". \n", + "\n", + "The author also notes that the focus of the article is on efforts to model language, as it is the work on language that is currently evoking the most speculation about the future of AI. The article then delves into the history of symbolic AI, starting from the 1950s when the term \"artificial intelligence\" was coined by John McCarthy. \n", + "\n", + "The article provides a comprehensive and thought-provoking view on the history and potential future of AI, particularly in the context of language modeling. \n", + "\n", + "Source: [LessWrong](https://www.lesswrong.com/posts/K9GP2ZSsugRAGfFns/a-dialectical-view-of-the-history-of-ai-part-1-we-re-only-in)\u001b[0m\n", + "\n", + "\u001b[1m> Finished chain.\u001b[0m\n" + ] + }, + { + "data": { + "application/vnd.google.colaboratory.intrinsic+json": { + "type": "string" + }, + "text/plain": [ + "'The article titled \"A dialectical view of the history of AI, Part 1: We’re only in the antithesis phase. [A synthesis is in the future.]\" on LessWrong provides a unique perspective on the evolution of AI. The author, Bill Benzon, uses the concept of dialectical change, primarily attributed to Hegel and Marx, to explain the history of AI. \\n\\nThe dialectical process is described as a movement from a thesis, to an antithesis, and finally, to a synthesis on a higher level. The author applies this concept to AI by identifying three eras. The first era, the THESIS, ran from the 1950s to the 1980s and was based on top-down deductive symbolic methods. The second era, the ANTITHESIS, began in the 1990s and is based on bottom-up statistical methods. These two eras are conceptually and computationally quite different. \\n\\nAs for the third era, the SYNTHESIS, the author speculates that it will involve a synthesis of conceptual ideas and computational techniques from the previous eras. However, it is unclear if this era will occur or if the current era will take us \"all the way\". \\n\\nThe author also notes that the focus of the article is on efforts to model language, as it is the work on language that is currently evoking the most speculation about the future of AI. The article then delves into the history of symbolic AI, starting from the 1950s when the term \"artificial intelligence\" was coined by John McCarthy. \\n\\nThe article provides a comprehensive and thought-provoking view on the history and potential future of AI, particularly in the context of language modeling. \\n\\nSource: [LessWrong](https://www.lesswrong.com/posts/K9GP2ZSsugRAGfFns/a-dialectical-view-of-the-history-of-ai-part-1-we-re-only-in)'" + ] + }, + "execution_count": 9, + "metadata": {}, + "output_type": "execute_result" + } + ], "source": [ - "from langchain.agents import AgentType, initialize_agent\n", - "from langchain.chat_models import ChatOpenAI\n", - "from langchain.tools import MetaphorSearchResults\n", - "\n", - "llm = ChatOpenAI(model_name=\"gpt-4\", temperature=0.7)\n", - "\n", - "metaphor_tool = MetaphorSearchResults(api_wrapper=search)\n", - "\n", - "agent_chain = initialize_agent(\n", - " [metaphor_tool, extract_text, navigate_tool],\n", - " llm,\n", - " agent=AgentType.STRUCTURED_CHAT_ZERO_SHOT_REACT_DESCRIPTION,\n", - " verbose=True,\n", - ")\n", - "\n", - "agent_chain.run(\n", - " \"find me an interesting tweet about AI safety using Metaphor, then tell me the first sentence in the post. Do not finish until able to retrieve the first sentence.\"\n", + "agent_executor.run(\n", + " \"Summarize for me an interesting article about AI from lesswrong.com published after October 2023.\"\n", ")" ] } ], "metadata": { + "colab": { + "provenance": [] + }, "kernelspec": { "display_name": "Python 3", "language": "python", @@ -425,5 +453,5 @@ } }, "nbformat": 4, - "nbformat_minor": 2 + "nbformat_minor": 0 }