responsible_ai/bedrock-guardrails/Apply_Guardrail_with_Streaming_and_Long_Context.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## ApplyGuardrail for Amazon Bedrock\n",
    "#### Examples using streaming and long context input\n",
    "---\n",
    "Guardrails for Amazon Bedrock evaluates user inputs and FM responses based on use case specific policies, and provides an additional layer of safeguards regardless of the underlying FM. Guardrails can be applied across all large language models (LLMs) on Amazon Bedrock, including fine-tuned models and even Generative AI applications outside of Amazon Bedrock. Customers can create multiple guardrails, each configured with a different combination of controls, and use these guardrails across different applications and use cases. Guardrails allows you to configure denied topics, filter harmful content, and remove sensitive information.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this notebook we will showcase how to use `ApplyGuardrail` API from Amazon Bedrock. This is an independent API to apply guardrails to any generative AI application both at input and output level.\n",
    "\n",
    "### Overview\n",
    "The new ApplyGuardrail API allows customers to assess any text using their pre-configured Bedrock Guardrails, without\n",
    "invoking the foundation models.\n",
    "Key Features:\n",
    "1. Content Validation: Send any text input or output to the `ApplyGuardrail` API to have it evaluated against your defined topic avoidance rules, content filters, word blocklists, PII detectors, regular expressions, profanity, and contextual grounding. You can evaluate user inputs and FM generated outputs independently.\n",
    "2. Flexible Deployment: Integrate the Guardrails API anywhere in your application flow to validate data before processing or serving results to users. E.g. For a RAG application, you can now evaluate the user input prior to performing the retrieval instead of waiting until the final response generation.\n",
    "3. ApplyGuardrail is decoupled from foundational models. You can now use Guardrails without invoking Foundation Models.\n",
    "You can use the assessment results to design the experience on your generative AI application."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Start by installing the dependencies to ensure we have a recent version\n",
    "!pip install --upgrade --force-reinstall boto3 botocore awscli\n",
    "import boto3\n",
    "print(boto3.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Guardrails in Amazon Bedrock is priced in the terms of a text unit. A text unit can contain up to 1000 characters. If a text input is more than 1000 characters, it is processed as multiple text units, each containing 1000 characters or less. For example, if a text input contains 5600 characters, it will be charged for 6 text units. Read more about Guardrails pricing here: https://aws.amazon.com/bedrock/pricing/\n",
    "\n",
    "`ApplyGuardrail` API has a default limit of 25 text units/second."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Let's specify the parameters needed for execution later\n",
    "\n",
    "REGION_NAME = \"us-east-1\"\n",
    "MODEL_ID = \"anthropic.claude-3-haiku-20240307-v1:0\"\n",
    "TEXT_UNIT = 1000 # characters\n",
    "LIMIT_TEXT_UNIT = 25"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "from textwrap import wrap\n",
    "import boto3\n",
    "\n",
    "# Make sure you have AWS credentials or AWS profile setup before running this cell\n",
    "bedrock_client = boto3.client(\"bedrock\", region_name=REGION_NAME)\n",
    "bedrock_runtime = boto3.client(\"bedrock-runtime\", region_name=REGION_NAME)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "### Creating a Guardrail\n",
    "\n",
    "Guardrails for Amazon Bedrock have multiple components which include Content Filters, Denied Topics, Word and Profanity Filters, and Sensitive Word (PII & Regex) Filters. For a full list check out the [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-create.html).\n",
    "\n",
    "We are creating a guardrail which prevents providing fiduciary advice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = bedrock_client.create_guardrail(\n",
    "    name='fiduciary-advice',\n",
    "    description='Prevents the our model from providing fiduciary advice.',\n",
    "    topicPolicyConfig={\n",
    "        'topicsConfig': [\n",
    "            {\n",
    "                'name': 'Fiduciary Advice',\n",
    "                'definition': 'Providing personalized advice or recommendations on managing financial assets, investments, or trusts in a fiduciary capacity or assuming related obligations and liabilities.',\n",
    "                'examples': [\n",
    "                    'What stocks should I invest in for my retirement?',\n",
    "                    'Is it a good idea to put my money in a mutual fund?',\n",
    "                    'How should I allocate my 401(k) investments?',\n",
    "                    'What type of trust fund should I set up for my children?',\n",
    "                    'Should I hire a financial advisor to manage my investments?'\n",
    "                ],\n",
    "                'type': 'DENY'\n",
    "            }\n",
    "        ]\n",
    "    },\n",
    "   contentPolicyConfig={\n",
    "    'filtersConfig': [\n",
    "        {\n",
    "            'type': 'SEXUAL',\n",
    "            'inputStrength': 'HIGH',\n",
    "            'outputStrength': 'HIGH'\n",
    "        },\n",
    "        {\n",
    "            'type': 'VIOLENCE',\n",
    "            'inputStrength': 'HIGH',\n",
    "            'outputStrength': 'HIGH'\n",
    "        },\n",
    "        {\n",
    "            'type': 'HATE',\n",
    "            'inputStrength': 'HIGH',\n",
    "            'outputStrength': 'HIGH'\n",
    "        },\n",
    "        {\n",
    "            'type': 'INSULTS',\n",
    "            'inputStrength': 'HIGH',\n",
    "            'outputStrength': 'HIGH'\n",
    "        },\n",
    "        {\n",
    "            'type': 'MISCONDUCT',\n",
    "            'inputStrength': 'HIGH',\n",
    "            'outputStrength': 'HIGH'\n",
    "        },\n",
    "        {\n",
    "            'type': 'PROMPT_ATTACK',\n",
    "            'inputStrength': 'HIGH',\n",
    "            'outputStrength': 'NONE'\n",
    "        }\n",
    "    ]\n",
    "\n",
    "    },\n",
    "wordPolicyConfig={\n",
    "    'wordsConfig': [\n",
    "        {\n",
    "            'text': 'fiduciary advice'\n",
    "        },\n",
    "        {\n",
    "            'text': 'investment recommendations'\n",
    "        },\n",
    "        {\n",
    "            'text': 'stock picks'\n",
    "        },\n",
    "        {\n",
    "            'text': 'financial planning guidance'\n",
    "        },\n",
    "        {\n",
    "            'text': 'portfolio allocation advice'\n",
    "        },\n",
    "        {\n",
    "            'text': 'retirement fund suggestions'\n",
    "        },\n",
    "        {\n",
    "            'text': 'wealth management tips'\n",
    "        },\n",
    "        {\n",
    "            'text': 'trust fund setup'\n",
    "        },\n",
    "        {\n",
    "            'text': 'investment strategy'\n",
    "        },\n",
    "        {\n",
    "            'text': 'financial advisor recommendations'\n",
    "        }\n",
    "    ],\n",
    "    'managedWordListsConfig': [\n",
    "        {\n",
    "            'type': 'PROFANITY'\n",
    "        }\n",
    "    ]\n",
    "},\n",
    "sensitiveInformationPolicyConfig={\n",
    "    'piiEntitiesConfig': [\n",
    "        {\n",
    "            'type': 'EMAIL',\n",
    "            'action': 'ANONYMIZE'\n",
    "        },\n",
    "        {\n",
    "            'type': 'PHONE',\n",
    "            'action': 'ANONYMIZE'\n",
    "        },\n",
    "        {\n",
    "            'type': 'NAME',\n",
    "            'action': 'ANONYMIZE'\n",
    "        },\n",
    "        {\n",
    "            'type': 'US_SOCIAL_SECURITY_NUMBER',\n",
    "            'action': 'BLOCK'\n",
    "        },\n",
    "        {\n",
    "            'type': 'US_BANK_ACCOUNT_NUMBER',\n",
    "            'action': 'BLOCK'\n",
    "        },\n",
    "        {\n",
    "            'type': 'CREDIT_DEBIT_CARD_NUMBER',\n",
    "            'action': 'BLOCK'\n",
    "        }\n",
    "    ],\n",
    "    'regexesConfig': [\n",
    "        {\n",
    "            'name': 'Account Number',\n",
    "            'description': 'Matches account numbers in the format XXXXXX1234',\n",
    "            'pattern': r'\\b\\d{6}\\d{4}\\b',\n",
    "            'action': 'ANONYMIZE'\n",
    "        }\n",
    "    ]\n",
    "},\n",
    "    blockedInputMessaging='I apologize, but I am not able to provide fiduciary advice. Additionally, it seems that you may have included some sensitive personal or financial information in your request. For your privacy and security, please modify your input and try again without including any personal, financial, or restricted details.',\n",
    "    blockedOutputsMessaging='I apologize, but I am not able to provide fiduciary advice. Additionally, it seems that you may have included some sensitive personal or financial information in your request. For your privacy and security, please modify your input and try again without including any personal, financial, or restricted details.',\n",
    ")\n",
    "print(response)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "guardrail_id = response['guardrailId']\n",
    "guardrail_version = response['version'] \n",
    "\n",
    "print(f\"Guardrail ID: {guardrail_id}\")\n",
    "print(f\"Guardrail Version: {guardrail_version}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Usage\n",
    "The `ApplyGuardrail` request allows customer to pass all their content that should be guarded using their defined Guardrail. The source field should be set to `INPUT` when the content to evaluated is from a user, typically the LLM prompt. The source should be set to `OUTPUT` when the model output Guardrail should be enforced, typically an LLM response.\n",
    "\n",
    "### Strategy\n",
    "We will showcase how you can apply guardrail in multiple scenarios\n",
    "1. Small input content (<25 text units)\n",
    "2. Large input content (>25 text units)\n",
    "3. Streaming LLM output\n",
    "\n",
    "If the content is larger than the quota limits of the `ApplyGuardrail` API then we will have to chunk the original content to smaller chunks to not hit throttling limit.\n",
    "\n",
    "Also, in the case of streaming the chunks could contain only a few tokens, it wouldn't be wise to apply guardrail on every new chunk neither it would be feasible to wait for the entire output to be generated to then apply guardrail. Instead in order to find the best fit we can apply guardrail whenever we have enough tokens i.e. tokens ~= 1 text unit, this will ensure both cost control as well as enough content available in the chunk to find potentially violating content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "def check_severe_violations(violations):\n",
    "    # When guardrail intervenes either the action on the request is BLOCKED or NONE\n",
    "    # Here we check how many of the violations lead to blocking the request\n",
    "    severe_violations = [violation['action']=='BLOCKED' for violation in violations]\n",
    "    return sum(severe_violations)\n",
    "\n",
    "def is_policy_assessement_blocked(assessments):\n",
    "    # While creating the guardrail you could specify multiple types of policies.\n",
    "    # At the time of assessment all the policies should be checked for potential violations\n",
    "    # If there is even 1 violation that blocks the request, the entire request is blocked\n",
    "    blocked = []\n",
    "    for assessment in assessments:\n",
    "        if 'topicPolicy' in assessment:\n",
    "            blocked.append(check_severe_violations(assessment['topicPolicy']['topics']))\n",
    "        if 'wordPolicy' in assessment:\n",
    "            if 'customWords' in assessment['wordPolicy']:\n",
    "                blocked.append(check_severe_violations(assessment['wordPolicy']['customWords']))\n",
    "            if 'managedWordLists' in assessment['wordPolicy']:\n",
    "                blocked.append(check_severe_violations(assessment['wordPolicy']['managedWordLists']))\n",
    "        if 'sensitiveInformationPolicy' in assessment:\n",
    "            if 'piiEntities' in assessment['sensitiveInformationPolicy']:\n",
    "                blocked.append(check_severe_violations(assessment['sensitiveInformationPolicy']['piiEntities']))\n",
    "            if 'regexes' in assessment['sensitiveInformationPolicy']:\n",
    "                blocked.append(check_severe_violations(assessment['sensitiveInformationPolicy']['regexes']))\n",
    "        if 'contentPolicy' in assessment:\n",
    "            blocked.append(check_severe_violations(assessment['contentPolicy']['filters']))\n",
    "    severe_violation_count = sum(blocked)\n",
    "    print(f'\\033[91m::Guardrail:: {severe_violation_count} severe violations detected\\033[0m')\n",
    "    return severe_violation_count>0\n",
    "\n",
    "def apply_guardrail(text, text_source_type, guardrail_id, guardrail_version=\"DRAFT\"):\n",
    "    print(f'\\n\\n\\033[91m::Guardrail:: Applying guardrail with {(len(text)//TEXT_UNIT)+1} text units\\033[0m\\n')\n",
    "    response = bedrock_runtime.apply_guardrail(\n",
    "        guardrailIdentifier=guardrail_id,\n",
    "        guardrailVersion=guardrail_version, \n",
    "        source=text_source_type, # can be 'INPUT' or 'OUTPUT'\n",
    "        content=[{\"text\": {\"text\": text}}]\n",
    "    )\n",
    "    if response['action'] == 'GUARDRAIL_INTERVENED':\n",
    "        is_blocked = is_policy_assessement_blocked(response['assessments'])\n",
    "        alternate_text = ' '.join([output['text'] for output in response['outputs']])\n",
    "        return is_blocked, alternate_text, response\n",
    "    else:\n",
    "        # Return the default response in case of no guardrail intervention\n",
    "        return False, text, response\n",
    "\n",
    "def apply_guardrail_full_text(text, text_source_type, guardrail_id, guardrail_version=\"DRAFT\"):\n",
    "    text_length = len(text)\n",
    "    filtered_text = ''\n",
    "    if text_length <= LIMIT_TEXT_UNIT*TEXT_UNIT:\n",
    "        return apply_guardrail(text, text_source_type, guardrail_id, guardrail_version)\n",
    "    else:\n",
    "        # If the text length is greater than the default text unit limits then it's better to chunk the text to avoid throttling.\n",
    "        for i, chunk in enumerate(wrap(text, LIMIT_TEXT_UNIT*TEXT_UNIT)):\n",
    "            print(f'::Guardrail::Applying guardrails at chunk {i+1}')\n",
    "            is_blocked, alternate_text, response = apply_guardrail(chunk, text_source_type, guardrail_id, guardrail_version)\n",
    "            if is_blocked:\n",
    "                filtered_text = alternate_text\n",
    "                break\n",
    "            # It could be the case that guardrails intervened and anonymized PII in the input text,\n",
    "            # we can then take the output from guardrails to create filtered text response.\n",
    "            filtered_text += alternate_text\n",
    "        return is_blocked, filtered_text, response"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from botocore.exceptions import ClientError\n",
    "\n",
    "def stream_conversation(messages,\n",
    "                        system_prompts,\n",
    "                        inference_config,\n",
    "                        additional_model_fields):\n",
    "    \n",
    "    response = bedrock_runtime.converse_stream(\n",
    "        modelId=MODEL_ID,\n",
    "        messages=messages,\n",
    "        system=system_prompts,\n",
    "        inferenceConfig=inference_config,\n",
    "        additionalModelRequestFields=additional_model_fields\n",
    "    )\n",
    "\n",
    "    stream = response.get('stream')\n",
    "    full_text = \"\"\n",
    "    buffer_text = \"\"\n",
    "    applied_guardrails = []\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            if 'messageStart' in event:\n",
    "                print(f\"\\nRole: {event['messageStart']['role']}\")\n",
    "\n",
    "            if 'contentBlockDelta' in event:\n",
    "                new_text = event['contentBlockDelta']['delta']['text']\n",
    "\n",
    "                if len(buffer_text + new_text) > TEXT_UNIT:\n",
    "                    is_blocked, alt_text, guardrail_response = apply_guardrail(buffer_text, \"OUTPUT\", guardrail_id, guardrail_version)\n",
    "                    if is_blocked:\n",
    "                        event['messageStop'] = {\n",
    "                            'stopReason': guardrail_response['action'], \n",
    "                            'output': alt_text,\n",
    "                            'assessments': guardrail_response['assessments'],\n",
    "                        }\n",
    "                        full_text = alt_text\n",
    "                    else:\n",
    "                        full_text += alt_text\n",
    "                    print(alt_text, end=\"\")\n",
    "                    applied_guardrails.append(guardrail_response)\n",
    "                    buffer_text = new_text\n",
    "                else: \n",
    "                    buffer_text += new_text\n",
    "\n",
    "            if 'messageStop' in event:\n",
    "                if event['messageStop']['stopReason'] == 'GUARDRAIL_INTERVENED':\n",
    "                    print(f\"\\nStop reason: {event['messageStop']['stopReason']}\")\n",
    "                    break\n",
    "                else:\n",
    "                    print(f\"\\nStop reason: {event['messageStop']['stopReason']}\")\n",
    "                    is_blocked, alt_text, guardrail_response = apply_guardrail(buffer_text, \"OUTPUT\", guardrail_id, guardrail_version)\n",
    "                    if is_blocked:\n",
    "                        print(alt_text)\n",
    "                        if 'metadata' not in event:\n",
    "                            event['metadata'] = {}\n",
    "                        event['metadata']['guardrails_usage'] = guardrail_response['usage']\n",
    "                        applied_guardrails.append(guardrail_response)\n",
    "\n",
    "            if 'metadata' in event:\n",
    "                metadata = event['metadata']\n",
    "                if 'usage' in metadata:\n",
    "                    print(\"\\nToken usage\")\n",
    "                    print(f\"Input tokens: {metadata['usage']['inputTokens']}\")\n",
    "                    print(\n",
    "                        f\":Output tokens: {metadata['usage']['outputTokens']}\")\n",
    "                    print(f\":Total tokens: {metadata['usage']['totalTokens']}\")\n",
    "                    print(f\":Total text units: {(len(full_text)//TEXT_UNIT)+1}\")\n",
    "                if 'metrics' in event['metadata']:\n",
    "                    print(\n",
    "                        f\"Latency: {metadata['metrics']['latencyMs']} milliseconds\")\n",
    "                if 'guardrails_usage' in event['metadata']:\n",
    "                    print(event['metadata']['guardrails_usage'])\n",
    "    return full_text, applied_guardrails\n",
    "\n",
    "def generate(input_message):\n",
    "\n",
    "    system_prompt = \"\"\"You are an assistant that helps with tasks from users. Be as elaborate as possible\"\"\"\n",
    "\n",
    "    message = {\n",
    "        \"role\": \"user\",\n",
    "        \"content\": [{\"text\": input_message}]\n",
    "    }\n",
    "    messages = [message]\n",
    "    \n",
    "    # System prompts.\n",
    "    system_prompts = [{\"text\" : system_prompt}]\n",
    "\n",
    "    # inference parameters to use.\n",
    "    temperature = 0.5\n",
    "\n",
    "    # Base inference parameters.\n",
    "    inference_config = {\n",
    "        \"temperature\": temperature\n",
    "    }\n",
    "    # Additional model inference parameters.\n",
    "    additional_model_fields = {}\n",
    "\n",
    "    try:\n",
    "        full_text, applied_guardrails = stream_conversation(messages, system_prompts, inference_config, additional_model_fields)\n",
    "    except ClientError as err:\n",
    "        message = err.response['Error']['Message']\n",
    "        print(\"A client error occured: \" +\n",
    "              format(message))\n",
    "\n",
    "    else:\n",
    "        print(\n",
    "            f\"Finished streaming messages with model {MODEL_ID}.\")\n",
    "    return full_text, applied_guardrails"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### OUTPUT - Streaming\n",
    "At first let's test the case where guardrail would intervene but the text generation will continue as normal."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_1 = \"List 3 names of prominent CEOs and later tell me what is a bank and what are the benefits of opening a savings account?\"\n",
    "full_text, applied_guardrails = generate(sample_1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can clearly see that the guardrail intervened above and annonmyized some names in the text generation. Let's examine what `assessments` were made by the guardrail."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for guardrail in applied_guardrails:\n",
    "    if guardrail['action']!='NONE':\n",
    "        print(f\"Guardrail Assessment: {guardrail['assessments']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above shows that `sensitiveInformationPolicy` got invoked anonymizing the names generated by the model and cleaning the output.\n",
    "\n",
    "Now, we can test a different scenario where the input contains request for a fudiciary advice and we can observe guardrail being applied."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sample_2 = \"Tell me about why financial independence is important and only at the very end ask the question if you can help me to invest after retirement?\"\n",
    "full_text, applied_guardrails = generate(sample_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can observe above that the guardrail intervention occured, now let's examine which policies were violated. For that we can examine `assessments`, which is part of the response from `ApplyGuardrail` API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for guardrail in applied_guardrails:\n",
    "    if guardrail['action']!='NONE':\n",
    "        print(f\"Guardrail Assessment: {guardrail['assessments']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It can be observed that the text generation led to a point where model tried giving fudiciary advice which goes against the `topicPolicy` configured at the time of guardrail creation, hence this output was denied and blocked. If a policy is enforced, the corresponding `action` could either be `BLOCKED` or `NONE` depending on severity, type and configuration of the policy.\n",
    "\n",
    "Read more about different types of components of a guardrail in Amazon Bedrock [here](https://docs.aws.amazon.com/bedrock/latest/userguide/guardrails-components.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### INPUT - Small Document\n",
    "We can now test how guardrail can be applied to a small document. We will use the [Amazon Shareholders Letter 2023](https://www.aboutamazon.com/news/company-news/amazon-ceo-andy-jassy-2023-letter-to-shareholders), this document doesn't include any text that should cause guardrail to intervene."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "letter = open('./data/shareholder_letter.txt', 'r').read()\n",
    "print(f\"Length of the document: {len(letter)} characters\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "blocked, new_text, guardrail_response = apply_guardrail_full_text(letter, \"INPUT\", guardrail_id, guardrail_version)\n",
    "print(f\"\\nBlocked by guardrail: {'Yes' if blocked else 'No'}\")\n",
    "if blocked:\n",
    "    print(f'Guardrail Output: {new_text}')\n",
    "elif guardrail_response=='GUARDRAIL_INTERVENED' and not blocked:\n",
    "    print(f'Filtered Text Snippet: {new_text[:5000]}')\n",
    "\n",
    "print(f\"Assessments: {guardrail_response['assessments']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### INPUT - Large Document\n",
    "Now we can test with a different document which contains a fictious financial story generated with the help of an LLM. To increase the length of the document we can combine the shareholders letter and the financial story. This will allow showcasing the capability to chunk the document and then apply guardrail at individual chunks."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "financial_story = open('./data/financial_story.txt', 'r').read()\n",
    "large_text = letter + financial_story\n",
    "print(f\"Length of the document: {len(large_text)} characters\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "blocked, new_text, guardrail_response = apply_guardrail_full_text(large_text, \"INPUT\", guardrail_id, guardrail_version)\n",
    "print(f\"\\nBlocked by guardrail: {'Yes' if blocked else 'No'}\")\n",
    "if blocked:\n",
    "    print(f'Guardrail Output: {new_text}')\n",
    "elif guardrail_response=='GUARDRAIL_INTERVENED' and not blocked:\n",
    "    print(f'Filtered Text Snippet: {new_text[:5000]}')\n",
    "\n",
    "print(f\"Assessments: {guardrail_response['assessments']}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Clean-up\n",
    "Before wrapping up let's clean the resources created and delete the guardrail created."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "bedrock_client.delete_guardrail(guardrailIdentifier=guardrail_id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wrap-up\n",
    "In this sample you learned the ability to use `ApplyGuardrail` independent API under Amazon Bedrock.\n",
    "\n",
    "Try yourself:\n",
    "- You can change the chunking strategy based on your content for both the streaming as well as full text scenario\n",
    "- Test different text lengths to call the API on to find best fit for price-performance\n",
    "- Try changing the prompts to see if the guardrail intervene at the time of text generation\n",
    "- Apply guardrail in your own application"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "AGR",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}