api_fundamentals.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyOaTCsci3ctpXxAt8DVfw7n",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/michellepace/anthropic-model-compare/blob/main/api_fundamentals.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "<font size=\"6\"><b>Notes — API Fundementals Anthropic</b></font>\n",
        "\n",
        "<b>Quick Reference</b>\n",
        "\n",
        "---\n",
        "\n",
        "Notes taken from Anthropic Github course <a href=\"https://github.com/anthropics/courses/tree/master/anthropic_api_fundamentals\" target=\"_blank\">API Fundementals</a>.\n",
        "\n",
        "Written in short form, things that stood out and for quick reference later.\n",
        "\n",
        "**Contents**\n",
        "\n",
        "1. <a href=\"#id-setup\">Setup</a>\n",
        "1. <a href=\"#id-message-parameter\">Message Parameter</a>\n",
        "1. <a href=\"#id-measure-models\">Measure Models</a>\n",
        "1. <a href=\"#id-model-parameters\">Model Parameters</a>\n",
        "1. <a href=\"#id-image-helper\">Image Helper</a>\n",
        "1. <a href=\"#id-streaming\">Streaming</a>"
      ],
      "metadata": {
        "id": "TkedSy3xxn1P"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"id-setup\"></a>\n",
        "# **SETUP**"
      ],
      "metadata": {
        "id": "JeoRMtin0geq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Libraries (VSCode)"
      ],
      "metadata": {
        "id": "zed5bhAz9j8h"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# import os\n",
        "# from anthropic import Anthropic, APIError\n",
        "\n",
        "# # Working with images\n",
        "# from IPython.display import Image, display\n",
        "# import base64\n",
        "# import httpx"
      ],
      "metadata": {
        "id": "b18LMaOL9nDU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Libraries (Google Colab)"
      ],
      "metadata": {
        "id": "2pNF-UZm1Z83"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "MGJkWfJYxMyp"
      },
      "outputs": [],
      "source": [
        "# Install first\n",
        "try:\n",
        "    !pip install --upgrade-strategy only-if-needed --quiet anthropic\n",
        "    print(\"Success! Libraries installed.\")\n",
        "except Exception as e:\n",
        "    print(f\"Installation error occured: {str(e)}\")\n",
        "\n",
        "# Import all\n",
        "try:\n",
        "    from google.colab import userdata   # To access your Colab Secret: ANTHROPIC_API_KEY\n",
        "    from anthropic import Anthropic, APIError\n",
        "    from IPython.display import Image, display # Images\n",
        "    import base64 # Images\n",
        "    import httpx # Images\n",
        "    print(\"Success! Libraries now imported too.\")\n",
        "except Exception as e:\n",
        "    print(f\"Importantion error occured: {str(e)}\")"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Models"
      ],
      "metadata": {
        "id": "wBbO7L5l4or6"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "HAIKU_3 = 'claude-3-haiku-20240307'\n",
        "HAIKU = 'claude-3-5-haiku-latest'\n",
        "SONNET = 'claude-3-5-sonnet-latest'\n",
        "OPUS = 'claude-3-opus-latest'\n",
        "\n",
        "default_model = HAIKU_3 # can change later\n",
        "\n",
        "print(f'Notebook default model is:\\n • {default_model}')"
      ],
      "metadata": {
        "id": "IE6DhWNj1izX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Set Anthropic API Key (VSCode)"
      ],
      "metadata": {
        "id": "qrq1X4vX99R8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# ANTHROPIC_CLIENT = Anthropic(\n",
        "#     api_key = os.environ.get('ANTHROPIC_API_KEY')\n",
        "# )"
      ],
      "metadata": {
        "id": "tVbIUJI5-A9_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "- Set Anthropic API Key (Colab)"
      ],
      "metadata": {
        "id": "J5bq0_DpA_TD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#@title API Key in Colab⭐ { display-mode: \"form\" }\n",
        "#@markdown **Step 1:** Create an [Anthropic API key](https://console.anthropic.com/settings/keys).<br>\n",
        "#@markdown **Step 2:** Then run this cell by clicking the 'play' icon just under the title.<br>\n",
        "#@markdown **Step 3:** you will be guided to setup a Colab secret if you don't have one.<br>\n",
        "#@markdown\n",
        "\n",
        "\n",
        "anthropic_api_secret_name = 'ANTHROPIC_API_KEY'  # @param {type: \"string\"}\n",
        "\n",
        "def test_anthropic_connection(anthropic_client: Anthropic) -> None:\n",
        "    my_test_prompt = \"Hello Claude. Are you there?!\"\n",
        "\n",
        "    try:\n",
        "        message = anthropic_client.messages.create(\n",
        "            model=default_model,\n",
        "            max_tokens=20,\n",
        "            messages=[{\"role\": \"user\", \"content\": my_test_prompt}]\n",
        "        )\n",
        "        print(\"Success!\\nYour Anthropic API key works.\")\n",
        "        print(f\"• PROMPT CLAUDE: {my_test_prompt}\")\n",
        "        print(f\"• CLAUDE REPLIED: {message.content[0].text}✅\")\n",
        "\n",
        "    except Exception as e:\n",
        "        print(f\"Connection test failed: {str(e)} 🛑\")\n",
        "        raise KeyboardInterrupt(\"Stopping execution\")\n",
        "\n",
        "\n",
        "def validate_anthropic_api_key_format(api_key):\n",
        "    if not api_key.startswith('sk-'):\n",
        "        raise ValueError(\"Anthropic API keys start with \\\"sk-\\\"\")\n",
        "    if ' ' in api_key:\n",
        "        raise ValueError(\"Anthropic API keys don't have white spaces.\")\n",
        "    if len(api_key) <= 100:\n",
        "        raise ValueError(\"Anthropic API keys are longer than 100 characters.\")\n",
        "\n",
        "\n",
        "def get_anthropic_api_key(secret_name):\n",
        "    try:\n",
        "        api_key = userdata.get(secret_name)\n",
        "        validate_anthropic_api_key_format(api_key)\n",
        "        print('Success!')\n",
        "        print(f'Your Colab secret was found: \"{secret_name}\"')\n",
        "        print(f'• If it holds a valid API Key, we can connect to Claude.\\n')\n",
        "        return api_key\n",
        "\n",
        "    except userdata.SecretNotFoundError:\n",
        "        print(f'🛑Error: Did not find your Colab secret: \"{secret_name}\".')\n",
        "        print(f' To fix:')\n",
        "        print(f' 1. Click the \"key\" icon on the left')\n",
        "        print(f' 2. Add new secret with name \"{secret_name}\"')\n",
        "        print(f' 3. Set value to an Anthropic API key')\n",
        "        print(f' 4. Rerun this block and follow next instructions')\n",
        "        print(f' About Colab secrets: https://bit.ly/4cad0v7 \\n')\n",
        "        raise\n",
        "    except userdata.NotebookAccessError:\n",
        "        print(f'🛑Error: Notebook access denied for Colab secret \"{secret_name}\"')\n",
        "        print(f' To fix:')\n",
        "        print(f' 1. Rerun this block and click \"Grant access\"')\n",
        "        print(f' About Colab secrets: https://bit.ly/4cad0v7')\n",
        "        raise\n",
        "    except ValueError as ve:\n",
        "        print(f'🛑Error: Invalid format, {str(ve)}')\n",
        "        print(f' To fix:')\n",
        "        print(f' 1. Click the \"key\" icon on the left of this Notebook')\n",
        "        print(f' 2. Delete \"{anthropic_api_secret_name}\"')\n",
        "        print(f' 4. Rerun this block and follow next instructions')\n",
        "        raise\n",
        "    except Exception as e:\n",
        "        print(f'🛑Error: An unexpected one.')\n",
        "        print(f' Please check:')\n",
        "        print(f' 1. \"{secret_name}\" secret exists in Colab (click \"key\" icon on the left)')\n",
        "        print(f' 2. Secret value is a valid Anthropic API key')\n",
        "        print(f' Get API key: https://console.anthropic.com/settings/keys')\n",
        "        print(f' About Colab secrets: https://bit.ly/4cad0v7')\n",
        "        raise\n",
        "\n",
        "\n",
        "# Usage\n",
        "MY_ANTHROPIC_API_KEY = get_anthropic_api_key(anthropic_api_secret_name)\n",
        "\n",
        "ANTHROPIC_CLIENT = Anthropic(\n",
        "    api_key=MY_ANTHROPIC_API_KEY,\n",
        "    max_retries=2,\n",
        "    timeout=10, # Timeout of the retry\n",
        ")\n",
        "\n",
        "test_anthropic_connection(ANTHROPIC_CLIENT)"
      ],
      "metadata": {
        "id": "nZEVaNccZrws"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"id-message-parameter\"></a>\n",
        "# **MESSAGES PARAM**"
      ],
      "metadata": {
        "id": "KTtphq8nBp2B"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Messages parameter is a list of one or more dictionaries, where each dictionary has two keys:\n",
        "\n",
        "`role`:\n",
        "- Either 'user\" or \"assistant\" (must alternate)\n",
        "\n",
        "`content`:\n",
        "- Can be a string (will be treated as single text content block).\n",
        "- Can be a list of content dictionaries, each with a \"type\" (e.g., \"text\" or \"image\") and the corresponding content."
      ],
      "metadata": {
        "id": "uGOMLPm0BrT2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "claude_response = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=100,\n",
        "    messages=[\n",
        "        {'role': 'user', 'content': 'Hello Claude, today is Monday.'},\n",
        "        {'role': 'assistant', 'content': 'Okay, got it. Today is Monday.'},\n",
        "        {'role': 'user', 'content': 'What day is it?'},\n",
        "    ]\n",
        ")\n",
        "\n",
        "print(claude_response.content[0].text)"
      ],
      "metadata": {
        "id": "lvBhl6RbBxW-"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Images"
      ],
      "metadata": {
        "id": "tfwu67AzB6zx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Anthropic Tip:\n",
        "> Just as with document-query placement, Claude works best when images come before text. Images placed after text or interpolated with text will still perform well, but if your use case allows it, we recommend an image-then-text structure. From Anthropic docs [here](https://docs.anthropic.com/en/docs/build-with-claude/vision#prompt-examples).\n",
        "\n",
        "* Get the image setup:"
      ],
      "metadata": {
        "id": "gfp__eoxB8EL"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from IPython.display import Image, display\n",
        "import base64\n",
        "import httpx\n",
        "\n",
        "dog_image_url = 'https://michellepace.github.io/anthropic-notes/images_mine/test_dog.jpg'\n",
        "image1_media_type = \"image/jpeg\"\n",
        "image1_data = base64.standard_b64encode(httpx.get(dog_image_url).content).decode(\"utf-8\")\n",
        "\n",
        "display(Image(url=dog_image_url))"
      ],
      "metadata": {
        "id": "SiaRdqvUCEns"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Use the image in a conversation"
      ],
      "metadata": {
        "id": "rmpFB6tZDLEo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "claude_response = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=100,\n",
        "    messages=[\n",
        "        {'role': 'user', 'content': 'Hello Claude, today is Monday.'},\n",
        "        {'role': 'assistant', 'content': 'Okay, got it. Today is Monday.'},\n",
        "        {'role': 'user', 'content': [\n",
        "            {'type': 'image', 'source': {'type': 'base64', 'media_type': image1_media_type, 'data': image1_data}},\n",
        "            {'type': 'text', 'text': 'Is the day in this image the same?'}\n",
        "        ]}\n",
        "    ]\n",
        ")\n",
        "\n",
        "display(Image(url=dog_image_url))\n",
        "print(claude_response.content[0].text)"
      ],
      "metadata": {
        "id": "7P5iN6sFDMbX"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Prefilling\n",
        "\n",
        "Same as above, but we prefill words into Claude's mouth on the last message in the conversation:"
      ],
      "metadata": {
        "id": "LKvRP91wDPYJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "prefil_words = 'In the image I see that the day is:'\n",
        "\n",
        "claude_response = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=100,\n",
        "    messages=[\n",
        "        {'role': 'user', 'content': 'Hello Claude, today is Monday.'},\n",
        "        {'role': 'assistant', 'content': 'Okay, got it. Today is Monday.'},\n",
        "        {'role': 'user', 'content': [\n",
        "            {'type': 'image', 'source': {'type': 'base64', 'media_type': image1_media_type, 'data': image1_data}},\n",
        "            {'type': 'text', 'text': 'Is the day in this image the same?'}\n",
        "        ]},\n",
        "        {'role': 'assistant', 'content': prefil_words}\n",
        "    ]\n",
        ")\n",
        "\n",
        "print(f'Words I put in Claude\\'s mouth:\\n• \"{prefil_words}\"\\n')\n",
        "print(f'Claude Response:\\n• \"{claude_response.content[0].text}\"\\n')\n",
        "print(f'Showing them together:\\n• \"{prefil_words + claude_response.content[0].text}\"')"
      ],
      "metadata": {
        "id": "lm_A3M7tDThr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Shot Examples"
      ],
      "metadata": {
        "id": "2kPx0CCXDcUn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### plain txt"
      ],
      "metadata": {
        "id": "dPGlkM3UDiBt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tweets = [\n",
        "    'I love pickles',\n",
        "    'I hate pickles',\n",
        "    'I eat pickles on Tuesdays',\n",
        "    'Just tried the new spicy pickles from @PickleCo, and my taste buds are doing a happy dance! 🌶️🥒 #pickleslove #spicyfood',\n",
        "]\n",
        "\n",
        "prompt = f\"\"\"For each tweet in this list {tweets}, classify its sentiment as positive, negative, or neutral.\n",
        "For each tweet, return the classification in the format: \"Tweet: [tweet text] | Sentiment: [sentiment]\"\n",
        "\n",
        "Examples:\n",
        "Tweet: I could not live without pickles!! | Sentiment: positive\n",
        "Tweet: If I just look at pickles I die. | Sentiment: negative\n",
        "Tweet: She puts chillies on my sandwich. | Sentiment: neutral\n",
        "\"\"\"\n",
        "\n",
        "reponse = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=1000,\n",
        "    temperature=1,\n",
        "    messages=[{\n",
        "        \"role\": \"user\",\n",
        "        \"content\": [{\"type\": \"text\", \"text\": prompt}]\n",
        "    }]\n",
        ")\n",
        "print(reponse.content[0].text)"
      ],
      "metadata": {
        "id": "bqgfIsefDeSi"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Claude loves XML\n",
        "\n",
        "Why use XML tags?\n",
        "- **Clarity**: Clearly separate different parts of your prompt and ensure your prompt is well structured.\n",
        "- **Accuracy**: Reduce errors caused by Claude misinterpreting parts of your prompt.\n",
        "- **Flexibility**: Easily find, add, remove, or modify parts of your prompt without rewriting everything.\n",
        "- **Parseability**: Having Claude use XML tags in its output makes it easier to extract specific parts of its response by post-processing.\n",
        "\n",
        "See Docs: <a href=\"https://docs.anthropic.com/en/docs/build-with-claude/prompt-engineering/use-xml-tags\" target=\"_blank\">Use XML tags to structure your prompts</a>"
      ],
      "metadata": {
        "id": "eFv2NDZlDlt2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "tweets = [\n",
        "    'I love pickles',\n",
        "    'I hate pickles',\n",
        "    'I will eat pickles on Tuesday only',\n",
        "    'Just tried the new spicy pickles, happy dance! 🌶️🥒 #pickleslove #spicyfood',\n",
        "]\n",
        "\n",
        "prompt = f\"\"\"For each tweet in this list {tweets}, classify its sentiment as positive, negative, or neutral. Format each response using the exact XML structure shown in the examples below.\n",
        "\n",
        "Examples:\n",
        "\n",
        "<tweet>\n",
        "    <text>If I look at chillies I die.</text>\n",
        "    <sentiment>negative</sentiment>\n",
        "</tweet>\n",
        "\n",
        "<tweet>\n",
        "    <text>My wife puts chillies on my sandwich.</text>\n",
        "    <sentiment>neutral</sentiment>\n",
        "</tweet>\n",
        "\"\"\"\n",
        "\n",
        "reponse = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=1000,\n",
        "    temperature=1,\n",
        "    messages=[{\"role\": \"user\", \"content\": [{\"type\": \"text\", \"text\": prompt}]}]\n",
        ")\n",
        "print(reponse.content[0].text)"
      ],
      "metadata": {
        "id": "BhdY5dqeDm4N"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"id-measure-models\"></a>\n",
        "# **MEASURE MODELS**"
      ],
      "metadata": {
        "id": "n_c_YcFGD5Bf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "> **GENERAL**\n",
        "\n",
        "Major factors to consider:\n",
        "\n",
        "* The model's capabilities — smartness (do first)\n",
        "* The model's latency — speed (effected by whose serving the model)\n",
        "* The model's cost — money (watch out for hidden costs)\n",
        "* Security (see AI Engineering, Chapter 5: <a href=\"https://learning.oreilly.com/library/view/ai-engineering/9781098166298/ch05.html#ch05a_defensive_prompt_engineering_1730156991196256\" target=\"_blank\">Defensive Prompt Engineering</a>)\n",
        "\n",
        "Simple comparision of speed, capability, cost\n",
        "* See article: <a href=\"https://ailearnlog.com/pushing-aside-the-bench-for-the-mark/\" target=\"_blank\">Pushing Aside the Bench for the Mark: Choosing an LLM</a>\n",
        "* See supporting code: <a href=\"https://colab.research.google.com/github/michellepace/anthropic-model-compare/blob/main/Anthropic_Model_Compare_(simple).ipynb\" target=\"_blank\">Anthropic_Model_Compare_(simple).ipynb</a>\n",
        "\n",
        "Notes on Speed:\n",
        "* Units: commonly seconds/token, but I like tokens/second.\n",
        "* Experienced by product: Speed = (output tokens ÷ execution duration).\n",
        "* Also effected by where you are getting the model served from (eg AWS or  Anthropic themselves via their API), your network, and current congestion!\n",
        "* Use sampling (don't just measure the prompt once)\n",
        "\n",
        "Notes on Capaibility\n",
        "* Easy to measure if it is a deterministic answer (eg math solution)\n",
        "* Likely to be non-deterministic: lots of different ways to evaluate\n",
        "* Either way, use sampling, run on different days - just like speed - see article <a href=\"https://ailearnlog.com/pushing-aside-the-bench-for-the-mark/#appendix-i\" target=\"_blank\">Appendix I</a>."
      ],
      "metadata": {
        "id": "5w0ayS-rESXv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "> **TIPS ANTHROPIC**\n",
        "\n",
        "Start Haiku light:\n",
        "> When experimenting, we often recommend starting with the Haiku model. Haiku is a lightweight and fast model that can serve as an excellent starting point for many applications. Its speed and cost-effectiveness make it an attractive option for initial experimentation and prototyping. In many use cases, Haiku proves to be perfectly capable of generating high-quality responses that meet the needs of the application. By starting with Haiku, you can quickly iterate on your application, test different prompts and configurations, and gauge the model's performance without incurring significant costs or latency. If you are unhappy with the responses, it's easy to \"upgrade\" to a model like Claude 3.5 Sonnet.\n",
        "\n",
        "❗❗❗ VERY NB — upgrade / switch\n",
        "> As you develop and refine your application, it's essential to set up a comprehensive suite of evaluations specific to your use case and prompts. These evaluations will serve as a benchmark to measure the performance of your chosen model and help you make informed decisions about potential upgrades.\n",
        "\n",
        "> By establishing a rigorous evaluation framework, you can objectively compare the performance of different models across your specific use case. This empirical evidence will guide your decision-making process and ensure that you select the model that best aligns with your application's needs."
      ],
      "metadata": {
        "id": "YQGuHyYwEWfD"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "> **CLAUDE TABLE**\n",
        "\n",
        "Anthropic comparsion table as of Feb 2025, see latest <a href=\"https://docs.anthropic.com/en/docs/about-claude/models#model-comparison-table\" target=\"_blank\">here</a>.\n",
        "\n",
        "| What | Claude 3.5 Sonnet | Claude 3.5 Haiku | Claude 3 Opus | Claude 3 Haiku |\n",
        "|:--- |:--- |:--- |:--- |:--- |\n",
        "| **Description** | Our most intelligent model | Our fastest model | Powerful model for highly complex tasks | Fastest and most compact model for near-instant responsiveness |\n",
        "| **Strengths** | Highest level of intelligence and capability | Intelligence at blazing speeds | Top-level intelligence, fluency, and understanding | Quick and accurate targeted performance |\n",
        "| **Multilingual** | Yes | Yes | Yes | Yes |\n",
        "| **Vision** | Yes | No | Yes | Yes |\n",
        "| **Message Batches API** | Yes | Yes | Yes | Yes |\n",
        "| **Comparative latency** | Fast | Fastest | Moderately fast | Fastest |\n",
        "| **Context window** | 200k tokens | 200k tokens | 200k tokens  | 200k tokens |\n",
        "| **Max output** | 8192 tokens | 8192 tokens | 4096 tokens | 4096 tokens  |\n",
        "| **Cost (Input / Output per )** | $3.00 / $15.00 | $0.80 / $4.00 | $15.00 / $75.00 | $0.25 / $1.25 |\n",
        "| **Training data cut-off** | Apr 2024 | July 2024 | Aug 2023 | Aug 2023 |"
      ],
      "metadata": {
        "id": "7X-NuEq7EaMd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"id-model-parameters\"></a>\n",
        "# **MODEL PARAMS**"
      ],
      "metadata": {
        "id": "PT35QDKMFHpN"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Basics (required)\n",
        "\n",
        "* `model`\n",
        "* `max_tokens`: does a hard cut\n",
        "* `messages`: (section: Messages Paramger)\n",
        "* `temperature`: (not required b/c defaults to 1)\n",
        "\n",
        "**Gets set like so:**\n",
        "\n",
        "```python\n",
        "reponse = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=\"claude-3-5-sonnet-20241022\",\n",
        "    max_tokens= ... etc.\n",
        ")\n",
        "```"
      ],
      "metadata": {
        "id": "RrGY205NFKyj"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## max_token\n",
        "\n",
        "Contain costs and latency, truncaptes even if response is not finished.\n",
        "\n",
        "I am unsure if it guides response in the same as \"Target length: 40-50 words\"\n",
        "\n",
        "* `max_tokens`: hard cut in action"
      ],
      "metadata": {
        "id": "w86qvyVrFPje"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "response = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=5,\n",
        "    messages=[\n",
        "        {'role': 'user', 'content': 'Write a poem'},\n",
        "    ]\n",
        ")\n",
        "print(f'Response was:\\n \"{response.content[0].text}\"\\n')\n",
        "print(f'Stop reason was:\\n {response.stop_reason}\\n')\n",
        "print(f'Output tokens was:\\n {response.usage.output_tokens}\\n')\n",
        "print('Michelle — natural endings stop reason is: \"end_turn\"')"
      ],
      "metadata": {
        "id": "YB70qZuLFh3Y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## stop_sequence\n",
        "-  text strings that cause Claude to immediately stop generating when encountered\n",
        "- The model will stop before generating the actual stop sequence itself\n",
        "- stop reason will be: \"stop_sequence\"\n",
        "- Useful for: ??"
      ],
      "metadata": {
        "id": "kBFTdK1GFyoc"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def generate_random_letters_3_times():\n",
        "    for i in range(3):\n",
        "        response = ANTHROPIC_CLIENT.messages.create(\n",
        "            model=default_model,\n",
        "            max_tokens=500,\n",
        "            messages=[{\"role\": \"user\", \"content\": \"generate a poem\"}],\n",
        "            stop_sequences=[\"b\", \"c\"]\n",
        "        )\n",
        "        print(f\"Response {i+1} stopped because {response.stop_reason}.  The stop sequence was {response.stop_sequence}\")\n",
        "\n",
        "\n",
        "# Usage:\n",
        "generate_random_letters_3_times()"
      ],
      "metadata": {
        "id": "AWwul1IDF24G"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Sample: top_p"
      ],
      "metadata": {
        "id": "LlDyEEcIIQ0Q"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "A model constructs its outputs through a process known as sampling. There's many different sample strategies: all aim to nudge models toward responses with specific attributes.\n",
        "\n",
        "**Why care about sampling?**\n",
        "- The right strategy helps generate responses suitable for application (eg creative vs predictable).\n",
        "- Generate responses that follow certain formats and constraints.\n",
        "\n",
        "**Common strategies**\n",
        "- `temperature`: changes fatness of next token probability distribution. That is,adjusts the probability distribution of ALL possible tokens.\n",
        "- `top_k`: cuts off the long tail (zeros out the probabilities for anything below the k’th token)... but how do you know what k to use?\n",
        "- `top_p`: makes it easier, cuts off by CDF - so dynamic. So if you set 0.8, it will cut off the 20% of tokens that fall out the CDF. ThT is, dynamically selects a subset of tokens based on cumulative probability\n",
        "\n",
        "**When to use `top_p` vs `temp`** — I don't know! ❓❓\n",
        "\n",
        "I am unsure, but Anthropic docs says use one or the other. Presumably because of interaction and so unpredictability. But that doesn't makes sense mathematically? Use Cases for `top_p` acccording to AI Engineering, Chip:\n",
        "- The scope of valid responses should vary based on context\n",
        "- You need contextually appropriate outputs\n",
        "- You need contextually appropriate responses but don't want to manually adjust parameters for each prompt\n",
        "\n",
        "<a href=\"https://michellepace.github.io/anthropic-notes/images_mine/photo_top_p.jpg\" target=\"_blank\">\n",
        "  <img src=\"https://michellepace.github.io/anthropic-notes/images_mine/photo_top_p.jpg\" width=\"400px\">\n",
        "</a>"
      ],
      "metadata": {
        "id": "dYqMInZ3F-UV"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Sample: temp\n",
        "When generating text, Claude **predicts the probability distribution of the next token**. The temperature parameter is used to manipulate this probability distribution before sampling the next token.\n",
        "- **LOW**: distribution more peaked on \"safe\" choices (math questions!)\n",
        "- **HIGH**: distribution fattens out (not centered on only highly probable generic choices - ie creative).\n",
        "- Range [0, 1] and default: 1\n",
        "\n",
        "<img src=\"https://michellepace.github.io/anthropic-notes/images_mine/temperature.jpg\"/>\n",
        "\n",
        "* Demo temprature:"
      ],
      "metadata": {
        "id": "VN6pwEwZINly"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def demonstrate_temperature(turns):\n",
        "    temperatures = [0, 1]\n",
        "    for temperature in temperatures:\n",
        "        print(f\"Prompting Claude {turns} times with temperature of {temperature}\")\n",
        "        print(\"================\")\n",
        "        for i in range(turns):\n",
        "            response = ANTHROPIC_CLIENT.messages.create(\n",
        "                model=default_model,\n",
        "                max_tokens=100,\n",
        "                messages=[{\"role\": \"user\", \"content\": \"What colour is the most magical elephant? Only respond with the colour.\"}],\n",
        "                temperature=temperature\n",
        "            )\n",
        "            print(f\"Response {i+1}: {response.content[0].text}\")\n",
        "        print()\n",
        "\n",
        "# Usage:\n",
        "demonstrate_temperature(4)"
      ],
      "metadata": {
        "id": "y5qdXaYsJVk7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## system prompt"
      ],
      "metadata": {
        "id": "gIVDPaohJacy"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Litrally just the text that gets prefixed to the top of the prompt. Sets the stage for the conversation: high-level instructions, defining its role, or providing background information that should inform its responses.\n",
        "\n",
        "Anthropic says use it for: tone, context role.\n",
        "- NOT: Detailed instructions, external input content (such as documents)\n",
        "- NOT: examples, they should go inside the first `User` turn for better results."
      ],
      "metadata": {
        "id": "1dFrQMjOJcSj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def demonstrate_system_prompt(turns):\n",
        "    top_p_values = [0.1, 0.7, 0.95]\n",
        "    for top_p in top_p_values:\n",
        "        print(f\"Prompting Claude {turns} times with top_p of {top_p}\")\n",
        "        print(\"================\")\n",
        "        for i in range(turns):\n",
        "            response = ANTHROPIC_CLIENT.messages.create(\n",
        "                model=default_model,\n",
        "                max_tokens=20,\n",
        "                system=\"You are an expert BMW salesperson. All magical elephants come in BMW car colours.\",\n",
        "                messages=[{\"role\": \"user\", \"content\": \"What colour is the most magical elephant? Only respond with the colour.\"}],\n",
        "                top_p=top_p,\n",
        "                # temperature=top_p,\n",
        "            )\n",
        "            print(f\"Response {i+1}: {response.content[0].text}\")\n",
        "        print()\n",
        "\n",
        "# Usage:\n",
        "demonstrate_system_prompt(8)"
      ],
      "metadata": {
        "id": "ZL7JLrr6JmIr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"id-image-helper\"></a>\n",
        "# **IMAGE HELPER**\n",
        "\n",
        "Cleaner way to work with images. Also for messages: nice to extract from API call."
      ],
      "metadata": {
        "id": "V50ZHhuFJwEb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Helper"
      ],
      "metadata": {
        "id": "tvrqUVnHLBkR"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import base64\n",
        "import mimetypes\n",
        "import httpx\n",
        "\n",
        "def create_image_message(image_url_or_path: str) -> dict:\n",
        "    \"\"\"Create an image message block for Anthropic API call.\n",
        "    Args: URL or local file path to image\n",
        "    Returns: Image message block for Anthropic API call\n",
        "    \"\"\"\n",
        "    if image_url_or_path.startswith('http'):\n",
        "        response = httpx.get(image_url_or_path)\n",
        "        binary_data = response.content\n",
        "        mime_type, _ = mimetypes.guess_type(image_url_or_path)\n",
        "    else:\n",
        "        with open(image_url_or_path, \"rb\") as image_file:\n",
        "            binary_data = image_file.read()\n",
        "        mime_type, _ = mimetypes.guess_type(image_url_or_path)\n",
        "\n",
        "    base64_string = base64.b64encode(binary_data).decode('utf-8')\n",
        "\n",
        "    return {\n",
        "        \"type\": \"image\",\n",
        "        \"source\": {\"type\": \"base64\", \"media_type\": mime_type, \"data\": base64_string}\n",
        "    }"
      ],
      "metadata": {
        "id": "un7a3Ml9Jqkf"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## messy msgs\n",
        "Usage like before but with function (see beginning of Notebook)"
      ],
      "metadata": {
        "id": "XfhTOw9KJ6VT"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Claude call\n",
        "claude_response = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=100,\n",
        "    messages=[\n",
        "        {'role': 'user', 'content': 'Hello Claude, today is Monday.'},\n",
        "        {'role': 'assistant', 'content': 'Okay, got it. Today is Monday.'},\n",
        "        {'role': 'user', 'content': [\n",
        "            create_image_message(dog_image_url),\n",
        "            {'type': 'text', 'text': 'Is the day in this image the same?'}\n",
        "        ]}\n",
        "    ]\n",
        ")\n",
        "\n",
        "# Show\n",
        "display(Image(url=dog_image_url))\n",
        "print(claude_response.content[0].text)"
      ],
      "metadata": {
        "id": "-4qrH8uuJ9MV"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## nice msgs\n",
        "Usage but more organised and clean:"
      ],
      "metadata": {
        "id": "C10pycEzJ_Mh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Show image\n",
        "pig_image_url = 'https://michellepace.github.io/anthropic-notes/images_mine/test_pig.jpg'\n",
        "display(Image(url=pig_image_url))\n",
        "\n",
        "# Get messages organised (outside API call)\n",
        "messages = [\n",
        "    {'role': 'user', 'content': 'Hello Claude, today is Monday.'},\n",
        "    {'role': 'assistant', 'content': 'Okay, got it. Today is Monday.'},\n",
        "    {'role': 'user', 'content': [\n",
        "        create_image_message(pig_image_url),\n",
        "        {'type': 'text', 'text': 'Is the day in this image the same?'}\n",
        "    ]}\n",
        "]\n",
        "\n",
        "# Make call to Claude\n",
        "claude_response = ANTHROPIC_CLIENT.messages.create(\n",
        "    model=default_model,\n",
        "    max_tokens=100,\n",
        "    messages=messages\n",
        ")\n",
        "\n",
        "# Show Claude response\n",
        "print(claude_response.content[0].text)"
      ],
      "metadata": {
        "id": "gdMJa0y5KB9O"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "<a name=\"id-streaming\"></a>\n",
        "# **STREAMING**\n",
        "\n",
        "* Improve user experience: see text as it's getting generated\n",
        "* You care about reducing \"time to first token\" (ie time until you see something)\n",
        "* Skip for now, come back later: <a href=\"https://github.com/anthropics/courses/blob/master/anthropic_api_fundamentals/05_Streaming.ipynb\" target=\"_blank\">05_Streaming.ipynb</a>"
      ],
      "metadata": {
        "id": "migZ4Yj3KGNI"
      }
    }
  ]
}