BedrockChatUI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "fce80d69-2321-49fc-b34c-bca443c76a2a",
   "metadata": {},
   "source": [
    "This notebook was tested in a `ml.t3.medium` instance and Sagemaker`Data Science 3` image Studio Notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18167e24-3077-4e66-af16-d1bce1f4643e",
   "metadata": {
    "tags": []
   },
   "source": [
    "<img src=\"images/chatbot2.png\" width=\"800\"/>\n",
    "\n",
    "This sample notebooks implements a general chatbot.\n",
    "Key functionalities include:\n",
    "1. Saving of Conversation History in DynamoDB\n",
    "2. Handling Document upload for various supported document format (PDF, JPG, CSV, EXCEL, PNG, TXT, JSON) by passing the document local or S3 path.\n",
    "3. Implementing various prompt template store locally (can also be stored in S3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d217fbc3-0e61-4cf6-b61a-b138e9dbec8f",
   "metadata": {},
   "source": [
    "Install required packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36bd7a32-aaeb-4762-b246-7c9b7f30791e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%%sh\n",
    "## This is required if you choose not to use Amazon Textract\n",
    "apt-get update\n",
    "apt-get install tesseract-ocr-all -y"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78c4b80a",
   "metadata": {},
   "source": [
    "IGNORE ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5146a09e-407e-44da-98b0-39bf89599065",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "%pip install anthropic\n",
    "%pip install s3fs -U\n",
    "%pip install pandas -U\n",
    "%pip install --force-reinstall amazon-textract-textractor==1.7.1\n",
    "%pip install textract openpyxl python-calamine\n",
    "%pip install pypdf2 pytesseract python-pptx python-docx pillow"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "39e5a8f1",
   "metadata": {},
   "source": [
    "**From the menu bar, go to Kernel > Restart Kernel to restart the notebook as best practice**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "9e2ba6ed-5037-456f-89f9-5dd978b78ac4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import boto3\n",
    "from anthropic import Anthropic\n",
    "from botocore.config import Config\n",
    "import shutil\n",
    "import os\n",
    "import pandas as pd\n",
    "import time\n",
    "import json\n",
    "import base64\n",
    "import io\n",
    "from python_calamine import CalamineWorkbook\n",
    "import re\n",
    "import numpy as np\n",
    "import openpyxl\n",
    "from openpyxl.cell import Cell\n",
    "from openpyxl.worksheet.cell_range import CellRange\n",
    "import uuid\n",
    "from pptx import Presentation\n",
    "from botocore.exceptions import ClientError\n",
    "from textractor import Textractor\n",
    "from textractor.visualizers.entitylist import EntityList\n",
    "from textractor.data.constants import TextractFeatures\n",
    "from textractor.data.text_linearization_config import TextLinearizationConfig\n",
    "import pytesseract\n",
    "from PIL import Image\n",
    "import PyPDF2\n",
    "import chardet\n",
    "from datetime import datetime    \n",
    "from docx import Document as DocxDocument\n",
    "from docx.oxml.text.paragraph import CT_P\n",
    "from docx.oxml.table import CT_Tbl\n",
    "from docx.document import Document\n",
    "from docx.table import _Cell, Table\n",
    "from docx.text.paragraph import Paragraph\n",
    "from docx.table import Table as DocxTable\n",
    "import concurrent.futures\n",
    "from functools import partial\n",
    "import csv\n",
    "import textract"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a615d7f-307f-4b79-a8d4-601e242906d7",
   "metadata": {},
   "source": [
    "### Configurable:\n",
    "- `DYNAMODB_TABLE`: The name of the DynamoDB table used for storing chat history. Change to an empty string if you want to store chat history locally.\n",
    "- `DYNAMODB_USER`: The user ID for the application.\n",
    "- `BUCKET`: The name of the S3 bucket used for caching documents and extracted text.\n",
    "- `CHAT_HISTORY_LENGTH`: The number of recent chat messages to load from the DynamoDB table.\n",
    "- `REGION`: The AWS region to interact with AWS services .\n",
    "- `LOAD_DOC_IN_ALL_CHAT_CONVO`: A boolean flag indicating whether to load documents extracted text in the chat history. If set to false only question and answer will be loaded in chat history. If set to True question and answer including all associated document extracted text will be loaded in chat history.\n",
    "- `S3_DOC_CACHE_PATH`: S3 path to store attached document if from local system\n",
    "- `TEXTRACT_RESULT_CACHE_PATH`: S3 path to cache extracted PDF and Images \n",
    "- `USE_TEXTRACT`: Boolean flag on whether to use amazon Textract for OCR or python libraries. If you do not have access to Amazon textract set to False. I recommend to use Amazon Textract if possible for better quality document OCR and processing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "54ad51c8-7eca-4482-a334-15c6358c8f71",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "DYNAMODB_TABLE=\"\" # Leave Empty if not using DynamoDb for Chat history else pass name for a DynamoDB table\n",
    "DYNAMODB_USER= \"test-user-sonnet19\" #Change to prefered user name if using DynamoDB for chat history storage\n",
    "SESSIONID=str(time.time())\n",
    "REGION=\"us-east-1\"\n",
    "chat_hist=[]\n",
    "BUCKET=\"BUCKET NAME\"\n",
    "S3_DOC_CACHE_PATH='chatbot_uploads'\n",
    "TEXTRACT_RESULT_CACHE_PATH=\"textract_output\"\n",
    "LOAD_DOC_IN_ALL_CHAT_CONVO=True\n",
    "CHAT_HISTORY_LENGTH=5\n",
    "USE_TEXTRACT=False #Change to True to use Amazon Textract\n",
    "LOCAL_CHAT_FILE_NAME = \"chat-history.json\" # Name of file to store chat history locally if not using DynamoDB\n",
    "\n",
    "\n",
    "DYNAMODB  = boto3.resource('dynamodb', region_name=REGION)\n",
    "dynamo=boto3.client('dynamodb', region_name=REGION)\n",
    "S3=boto3.client('s3',region_name=REGION)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed85ee4d-1acf-42d1-906b-8b9f3f69ec4e",
   "metadata": {},
   "source": [
    "#### Initialize Bedrock Runtime"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "691af9ab-14a5-428f-9b17-89a785b2b401",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Create the bedrock runtime to invoke LLM\n",
    "from botocore.config import Config\n",
    "config = Config(\n",
    "    read_timeout=600, # Read timeout parameter\n",
    "    retries = dict(\n",
    "        max_attempts = 10 ## Handle retries\n",
    "    )\n",
    ")\n",
    "import boto3\n",
    "bedrock_runtime = boto3.client(service_name='bedrock-runtime',region_name=REGION,config=config)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c2d49c0-58ae-4984-8c19-34c0e659e4cc",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Create DynamoDB Table\n",
    "A DynamoDB Table is created with a user ID as partition Key and Session ID as sort key. \n",
    "This enables saving multiple chat session history under the same user id.\\\n",
    "Provide a bucket name that would be used to cache Amazon Textract results for document OCR."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "12996c34-511b-430b-8b73-e9ca8b838fe6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "if DYNAMODB_TABLE:\n",
    "    try:\n",
    "        table = DYNAMODB.create_table(\n",
    "            TableName=DYNAMODB_TABLE,\n",
    "            KeySchema=[\n",
    "                {\n",
    "                    'AttributeName': 'UserId',  # Partition key\n",
    "                    'KeyType': 'HASH'  \n",
    "                },\n",
    "                {\n",
    "                    'AttributeName': 'SessionId',   # Sort key\n",
    "                    'KeyType': 'RANGE'\n",
    "                }\n",
    "            ],\n",
    "            AttributeDefinitions=[\n",
    "                {\n",
    "                    'AttributeName': 'UserId',\n",
    "                    'AttributeType': 'S'   # String data type\n",
    "                },\n",
    "                {\n",
    "                    'AttributeName': 'SessionId',\n",
    "                    'AttributeType': 'S'\n",
    "                },\n",
    "            ],\n",
    "            BillingMode='PAY_PER_REQUEST'  # On-demand billing\n",
    "        )\n",
    "\n",
    "        print(\"Table status:\", table.table_status)\n",
    "\n",
    "        # Wait until the table exists.\n",
    "        table.meta.client.get_waiter(\"table_exists\").wait(TableName=\"SessionChatHistory\")\n",
    "        print(table.item_count)\n",
    "    except dynamo.exceptions.ResourceInUseException as e:\n",
    "        print(e.response['Error']['Message'])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5724ecf2-954f-4571-aa11-0b98558dffbc",
   "metadata": {},
   "source": [
    "## Utility Functions"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05af03f8-f817-4d2f-829d-47fd821e5f06",
   "metadata": {},
   "source": [
    "This function reads an Excel file from the specified S3 bucket using the provided S3 URI.\n",
    "   It loads the workbook using openpyxl `table_parser_openpyxl` or python-calamine `calamaine_excel_engine` (some excel files are better read with calamine as openpyxl does not work), unmerges any merged cells, and copies their values\n",
    "   to individual cells for every worksheet by calling the ` table_parser_utils` function. The worksheet data is then converted to a pandas DataFrame, and the\n",
    "   `strip_newline` function is applied to each cell value to remove newline characters.\n",
    "   Finally, the DataFrame is converted to a CSV string with pipe (|) as the delimiter and\n",
    "   returned.\n",
    "\n",
    "NOTE: The `calamaine_excel_engine` does not handle merged cells.\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "27929b3a-eae7-4abe-be72-f6932fb36184",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def strip_newline(cell):\n",
    "    return str(cell).strip()\n",
    "\n",
    "def table_parser_openpyxl(file):\n",
    "    # Read from S3\n",
    "    s3 = boto3.client('s3', region_name=REGION)\n",
    "    match = re.match(\"s3://(.+?)/(.+)\", file)\n",
    "    if match:\n",
    "        bucket_name = match.group(1)\n",
    "        key = match.group(2)\n",
    "        obj = s3.get_object(Bucket=bucket_name, Key=key)    \n",
    "        # Read Excel file from S3 into a buffer\n",
    "        xlsx_buffer = io.BytesIO(obj['Body'].read())\n",
    "        xlsx_buffer.seek(0)    \n",
    "        # Load workbook\n",
    "        wb = openpyxl.load_workbook(xlsx_buffer)    \n",
    "        all_sheets_string=\"\"\n",
    "        # Iterate over each sheet in the workbook\n",
    "        for sheet_name in wb.sheetnames:\n",
    "            # all_sheets_name.append(sheet_name)\n",
    "            worksheet = wb[sheet_name]\n",
    "\n",
    "            all_merged_cell_ranges: list[CellRange] = list(\n",
    "                worksheet.merged_cells.ranges\n",
    "            )\n",
    "            for merged_cell_range in all_merged_cell_ranges:\n",
    "                merged_cell: Cell = merged_cell_range.start_cell\n",
    "                worksheet.unmerge_cells(range_string=merged_cell_range.coord)\n",
    "                for row_index, col_index in merged_cell_range.cells:\n",
    "                    cell: Cell = worksheet.cell(row=row_index, column=col_index)\n",
    "                    cell.value = merged_cell.value        \n",
    "            # Convert sheet data to a DataFrame\n",
    "            df = pd.DataFrame(worksheet.values)\n",
    "            df = df.map(strip_newline)\n",
    "            # Convert to string and tag by sheet name\n",
    "            all_sheets_string+=f'<{sheet_name}>\\n{df.to_csv(sep=\"|\", index=False, header=0)}\\n</{sheet_name}>\\n'\n",
    "        return all_sheets_string\n",
    "    else:\n",
    "        raise Exception(f\"{file} not formatted as an S3 path\")\n",
    "\n",
    "def calamaine_excel_engine(file):\n",
    "    # # Read from S3\n",
    "    s3 = boto3.client('s3',region_name=REGION)\n",
    "    match = re.match(\"s3://(.+?)/(.+)\", file)\n",
    "    if match:\n",
    "        bucket_name = match.group(1)\n",
    "        key = match.group(2)\n",
    "        obj = s3.get_object(Bucket=bucket_name, Key=key)    \n",
    "        # Read Excel file from S3 into a buffer\n",
    "        xlsx_buffer = io.BytesIO(obj['Body'].read())\n",
    "        xlsx_buffer.seek(0)    \n",
    "        all_sheets_string=\"\"\n",
    "        # Load the Excel file\n",
    "        workbook = CalamineWorkbook.from_filelike(xlsx_buffer)\n",
    "        # Iterate over each sheet in the workbook\n",
    "        for sheet_name in workbook.sheet_names:\n",
    "            # Get the sheet by name\n",
    "            sheet = workbook.get_sheet_by_name(sheet_name)\n",
    "            df = pd.DataFrame(sheet.to_python(skip_empty_area=False))\n",
    "            df = df.map(strip_newline)\n",
    "            all_sheets_string+=f'<{sheet_name}>\\n{df.to_csv(sep=\"|\", index=False, header=0)}\\n</{sheet_name}>\\n'\n",
    "        return all_sheets_string\n",
    "    else:\n",
    "        raise Exception(f\"{file} not formatted as an S3 path\")\n",
    "\n",
    "def table_parser_utills(file):\n",
    "    try:\n",
    "        response= table_parser_openpyxl(file)\n",
    "        if response:\n",
    "            return response\n",
    "        else:\n",
    "            return calamaine_excel_engine(file)        \n",
    "    except Exception as e:\n",
    "        try:\n",
    "            return calamaine_excel_engine(file)\n",
    "        except Exception as e:\n",
    "            raise Exception(str(e))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb86bd7e-c44d-4496-bc5f-62f34b171e63",
   "metadata": {},
   "source": [
    "1. `get_s3_keys(prefix)`: Retrieves a list of object keys from an S3 bucket that match the specified prefix. It returns an empty string if no objects are found.\n",
    "2. `get_object_with_retry(bucket, key)`: Retrieves an object from an S3 bucket with retry functionality. It attempts to get the object and handles the \"DecryptionFailureException\" error by retrying with exponential backoff. If the maximum number of retries is exceeded, it raises an exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "4c88b302-d587-4710-847f-a7e334eefde5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_s3_keys(prefix):\n",
    "    \"\"\"list all keys in an s3 path\"\"\"\n",
    "    s3 = boto3.client('s3',region_name=REGION)\n",
    "    keys = []\n",
    "    next_token = None\n",
    "\n",
    "    while True:\n",
    "        if next_token:\n",
    "            response = s3.list_objects_v2(Bucket=BUCKET, Prefix=prefix, ContinuationToken=next_token)\n",
    "        else:\n",
    "            response = s3.list_objects_v2(Bucket=BUCKET, Prefix=prefix)\n",
    "\n",
    "        if \"Contents\" in response:\n",
    "            for obj in response['Contents']:\n",
    "                key = obj['Key']\n",
    "                name = key[len(prefix):]\n",
    "                keys.append(name)\n",
    "\n",
    "        if \"NextContinuationToken\" in response:\n",
    "            next_token = response[\"NextContinuationToken\"]\n",
    "        else:\n",
    "            break\n",
    "\n",
    "    return keys\n",
    "\n",
    "def get_object_with_retry(bucket, key):\n",
    "    \"\"\"Get object from s3 with error handling and retries\"\"\"\n",
    "    max_retries=5\n",
    "    initial_backoff=1\n",
    "    retries = 0\n",
    "    backoff = initial_backoff\n",
    "    s3 = boto3.client('s3',region_name=REGION)\n",
    "\n",
    "    while retries < max_retries:\n",
    "        try:\n",
    "            response = s3.get_object(Bucket=bucket, Key=key)\n",
    "            return response\n",
    "        except ClientError as e:\n",
    "            error_code = e.response['Error']['Code']\n",
    "            if error_code == 'DecryptionFailureException':\n",
    "                print(f\"Decryption failed, retrying in {backoff} seconds...\")\n",
    "                time.sleep(backoff)\n",
    "                backoff *= 2  # Exponential backoff\n",
    "                retries += 1\n",
    "            else:\n",
    "                raise e\n",
    "    # If we reach this point, it means the maximum number of retries has been exceeded\n",
    "    raise Exception(f\"Failed to get object {key} from bucket {bucket} after {max_retries} retries.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f56bf29-ed5f-4c66-b167-33a81f76da39",
   "metadata": {},
   "source": [
    "Extracts text from a PDF or image file using AWS Textract or Python Lib (Pypdf2 or PyTesseract).\n",
    "\n",
    "It checks if the extracted document content is already cached in S3 based on the file name. If found, it retrieves the cached text using the get_object_with_retry function.\n",
    "If the content is not cached and the USE_TEXTRACT flag is set:\n",
    "\n",
    "For PDF files, it uses the Textractor class from the textractor library to perform an asynchronous document analysis. It extracts text, layout, and tables from the PDF.\n",
    "For other file types, it uses the Textractor class to perform a synchronous document analysis.\n",
    "The extracted content is then uploaded to S3 for caching.\n",
    "\n",
    "\n",
    "If the USE_TEXTRACT flag is not set:\n",
    "\n",
    "For PDF files, it downloads the file from S3 using s3.Bucket(bucket_name).download_fileobj, reads the PDF using PyPDF2, and extracts text from each page.\n",
    "For image files, it downloads the file from S3, opens it using Image.open, and uses pytesseract to extract text from the image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "a65ff2ff-c800-40c4-812b-60112b847e4b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def exract_pdf_text_aws(file):    \n",
    "    file_base_name=os.path.basename(file)\n",
    "    dir_name, ext = os.path.splitext(file)\n",
    "    # Checking if extracted doc content is in S3\n",
    "    if USE_TEXTRACT:        \n",
    "        if [x for x in get_s3_keys(f\"{TEXTRACT_RESULT_CACHE_PATH}/\") if file_base_name in x]:    \n",
    "            response = get_object_with_retry(BUCKET, f\"{TEXTRACT_RESULT_CACHE_PATH}/{file_base_name}.txt\")#S3.get_object(Bucket=BUCKET, Key=f\"{TEXTRACT_RESULT_CACHE_PATH}/{file_base_name}.txt\")\n",
    "            text = response['Body'].read().decode()\n",
    "            return text\n",
    "        else:\n",
    "            \n",
    "            extractor = Textractor(region_name=\"us-east-1\")\n",
    "            # Asynchronous call, you will experience some wait time. Try caching results for better experience\n",
    "            if \"pdf\" in ext:\n",
    "                print(\"Asynchronous call, you may experience some wait time.\")\n",
    "                document = extractor.start_document_analysis(\n",
    "                file_source=file,\n",
    "                features=[TextractFeatures.LAYOUT,TextractFeatures.TABLES],       \n",
    "                save_image=False,   \n",
    "                s3_output_path=f\"s3://{BUCKET}/textract_output/\"\n",
    "            )\n",
    "            #Synchronous call\n",
    "            else:\n",
    "                document = extractor.analyze_document(\n",
    "                file_source=file,\n",
    "                features=[TextractFeatures.LAYOUT,TextractFeatures.TABLES],  \n",
    "                save_image=False,\n",
    "            )\n",
    "            config = TextLinearizationConfig(\n",
    "            hide_figure_layout=False,   \n",
    "            hide_header_layout=False,    \n",
    "            table_prefix=\"<table>\",\n",
    "            table_suffix=\"</table>\",\n",
    "            )\n",
    "            # Upload extracted content to s3\n",
    "            S3.put_object(Body=document.get_text(config=config), Bucket=BUCKET, Key=f\"{TEXTRACT_RESULT_CACHE_PATH}/{file_base_name}.txt\") \n",
    "            return document.get_text(config=config)\n",
    "    else:\n",
    "        s3=boto3.resource(\"s3\",region_name=REGION)\n",
    "        match = re.match(\"s3://(.+?)/(.+)\", file)\n",
    "        if match:\n",
    "            bucket_name = match.group(1)\n",
    "            key = match.group(2)    \n",
    "   \n",
    "        if \"pdf\" in ext:            \n",
    "            pdf_bytes = io.BytesIO()\n",
    "            \n",
    "            s3.Bucket(bucket_name).download_fileobj(key, pdf_bytes)\n",
    "            # Read the PDF from the BytesIO object\n",
    "            pdf_bytes.seek(0)                      \n",
    "            # Create a PDF reader object\n",
    "            pdf_reader = PyPDF2.PdfReader(pdf_bytes)\n",
    "            # Get the number of pages in the PDF\n",
    "            num_pages = len(pdf_reader.pages)\n",
    "            # Extract text from each page\n",
    "            text = ''\n",
    "            for page_num in range(num_pages):\n",
    "                page = pdf_reader.pages[page_num]\n",
    "                text += page.extract_text()\n",
    "        else:\n",
    "            img_bytes = io.BytesIO()\n",
    "            s3.Bucket(bucket_name).download_fileobj(key, img_bytes)\n",
    "            img_bytes.seek(0)\n",
    "            image = Image.open(img_bytes)\n",
    "            text = pytesseract.image_to_string(image)\n",
    "        return text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "75127a3e-55f8-44a0-9e29-f2c5f6891e47",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_s3_obj_from_bucket_(file):\n",
    "    \"\"\"Retrieves an object from an S3 bucket given its S3 URI.\n",
    "    Args:\n",
    "       file (str): The S3 URI of the object to retrieve, in the format \"s3://{bucket_name}/{key}\".\n",
    "   Returns:\n",
    "       botocore.response.StreamingBody: The retrieved S3 object.\n",
    "    \"\"\"\n",
    "    s3 = boto3.client('s3',region_name=REGION)\n",
    "    match = re.match(\"s3://(.+?)/(.+)\", file)\n",
    "    if match:\n",
    "        bucket_name = match.group(1)\n",
    "        key = match.group(2)    \n",
    "        obj = s3.get_object(Bucket=bucket_name, Key=key)  \n",
    "    return obj\n",
    "\n",
    "def put_obj_in_s3_bucket_(docs):\n",
    "    \"\"\"Uploads a file to an S3 bucket and returns the S3 URI of the uploaded object.\n",
    "    Args:\n",
    "       docs (str): The local file path of the file to upload to S3.\n",
    "   Returns:\n",
    "       str: The S3 URI of the uploaded object, in the format \"s3://{bucket_name}/{file_path}\".\n",
    "    \"\"\"\n",
    "    file_name=os.path.basename(docs)\n",
    "    file_path=f\"{S3_DOC_CACHE_PATH}/{file_name}\"\n",
    "    S3.upload_file(docs, BUCKET, file_path)\n",
    "    return f\"s3://{BUCKET}/{file_path}\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d32ce327-dff3-4414-abbd-5ad21bf7f8b6",
   "metadata": {},
   "source": [
    "The `process_files` function processes multiple attached files concurrently using a process pool executor. It takes a list of files, submits tasks to the executor to process each file using the `handle_doc_upload_or_s3` function, and collects the results and errors. It returns a tuple containing the processed results, errors, and a formatted result string. The function leverages concurrent processing to improve efficiency when handling a large number of files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "d40c45ee-14dc-4277-a0a4-0d9f98aba416",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def process_files(files):\n",
    "    results = []\n",
    "    result_string=\"\"\n",
    "    errors = []\n",
    "    future_proxy_mapping = {} \n",
    "    futures = []\n",
    "\n",
    "    with concurrent.futures.ProcessPoolExecutor() as executor:\n",
    "        # Partial function to pass the handle_doc_upload_or_s3 function\n",
    "        func = partial(handle_doc_upload_or_s3)   \n",
    "        for file in files:\n",
    "            future = executor.submit(func, file)\n",
    "            future_proxy_mapping[future] = file\n",
    "            futures.append(future)\n",
    "\n",
    "        # Collect the results and handle exceptions\n",
    "        for future in concurrent.futures.as_completed(futures):        \n",
    "            file_url= future_proxy_mapping[future]\n",
    "            try:\n",
    "                result = future.result()\n",
    "                results.append(result)\n",
    "                doc_name=os.path.basename(file_url)\n",
    "                \n",
    "                result_string+=f\"<{doc_name}>\\n{result}\\n</{doc_name}>\\n\"\n",
    "            except Exception as e:\n",
    "                # Get the original function arguments from the Future object\n",
    "                error = {'file': file_url, 'error': str(e)}\n",
    "                errors.append(error)\n",
    "\n",
    "    return results, errors, result_string"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66382201-5906-4a55-bf13-ca5e871f8815",
   "metadata": {},
   "source": [
    "- `extract_text_and_tables(docx_path)`: Extracts text and tables from a Word document (docx) file. It uses the python-docx library to iterate over the block-level items in the document. It identifies headings, lists, and tables based on their styles and tags them accordingly in the extracted content. It also handles nested tables by recursively parsing them.\n",
    "- `extract_text_from_pptx_s3(pptx_buffer)`: Extracts text from a PowerPoint presentation (pptx) file stored in S3. It takes a BytesIO buffer containing the pptx file content and uses the python-pptx library to extract text from each slide. It returns the extracted text as a single string.\n",
    "- `parse_csv_from_s3(s3_uri)`: Parses a CSV file stored in S3 using pandas. It detects the file encoding using `detect_encoding()`, sniffs the delimiter, and reads the CSV file into a pandas DataFrame. If an error occurs during parsing, it raises an InvalidContentError exception."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "abb2e468-9fca-4aac-8f0e-b176d939a63e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "class InvalidContentError(Exception):\n",
    "    pass\n",
    "\n",
    "def detect_encoding(s3_uri):\n",
    "    \"\"\"detect csv encoding\"\"\"\n",
    "    s3 = boto3.client('s3',region_name=REGION)\n",
    "    match = re.match(\"s3://(.+?)/(.+)\", s3_uri)\n",
    "    if match:\n",
    "        bucket_name = match.group(1)\n",
    "        key = match.group(2) \n",
    "    response = s3.get_object(Bucket=bucket_name, Key=key)\n",
    "    content = response['Body'].read()\n",
    "    result = chardet.detect(content)\n",
    "    return result['encoding']\n",
    "\n",
    "def parse_csv_from_s3(s3_uri):\n",
    "    \"\"\"read csv files\"\"\"\n",
    "    try:\n",
    "        # Detect the file encoding using chardet\n",
    "        encoding = detect_encoding(s3_uri)        \n",
    "        # Sniff the delimiter and read the CSV file\n",
    "        df = pd.read_csv(s3_uri, delimiter=None, engine='python', encoding=encoding)\n",
    "        return df.to_csv(index=False, sep=\"|\")\n",
    "    except Exception as e:\n",
    "        raise InvalidContentError(f\"Error: {e}\")\n",
    "    \n",
    "def iter_block_items(parent):\n",
    "    if isinstance(parent, Document):\n",
    "        parent_elm = parent.element.body\n",
    "    elif isinstance(parent, _Cell):\n",
    "        parent_elm = parent._tc\n",
    "    else:\n",
    "        raise ValueError(\"something's not right\")\n",
    "\n",
    "    for child in parent_elm.iterchildren():\n",
    "        if isinstance(child, CT_P):\n",
    "            yield Paragraph(child, parent)\n",
    "        elif isinstance(child, CT_Tbl):\n",
    "            yield DocxTable(child, parent)\n",
    "\n",
    "def extract_text_and_tables(docx_path):\n",
    "    \"\"\" Extract text from docx files\"\"\"\n",
    "    document = DocxDocument(docx_path)\n",
    "    content = \"\"\n",
    "    current_section = \"\"\n",
    "    section_type = None\n",
    "    for block in iter_block_items(document):\n",
    "        if isinstance(block, Paragraph):\n",
    "            if block.text:\n",
    "                if block.style.name == 'Heading 1':\n",
    "                    # Close the current section if it exists\n",
    "                    if current_section:\n",
    "                        content += f\"{current_section}</{section_type}>\\n\"\n",
    "                        current_section = \"\"\n",
    "                        section_type = None  \n",
    "                    section_type =\"h1\"\n",
    "                    content += f\"<{section_type}>{block.text}</{section_type}>\\n\"\n",
    "                elif block.style.name== 'Heading 3':\n",
    "                    # Close the current section if it exists\n",
    "                    if current_section:\n",
    "                        content += f\"{current_section}</{section_type}>\\n\"\n",
    "                        current_section = \"\"\n",
    "                    section_type = \"h3\"  \n",
    "                    content += f\"<{section_type}>{block.text}</{section_type}>\\n\"\n",
    "                \n",
    "                elif block.style.name == 'List Paragraph':\n",
    "                    # Add to the current list section\n",
    "                    if section_type != \"list\":\n",
    "                        # Close the current section if it exists\n",
    "                        if current_section:\n",
    "                            content += f\"{current_section}</{section_type}>\\n\"\n",
    "                        section_type = \"list\"\n",
    "                        current_section = \"<list>\"\n",
    "                    current_section += f\"{block.text}\\n\"\n",
    "                elif block.style.name.startswith('toc'):\n",
    "                    # Add to the current toc section\n",
    "                    if section_type != \"toc\":\n",
    "                        # Close the current section if it exists\n",
    "                        if current_section:\n",
    "                            content += f\"{current_section}</{section_type}>\\n\"\n",
    "                        section_type = \"toc\"\n",
    "                        current_section = \"<toc>\"\n",
    "                    current_section += f\"{block.text}\\n\"\n",
    "                else:\n",
    "                    # Close the current section if it exists\n",
    "                    if current_section:\n",
    "                        content += f\"{current_section}</{section_type}>\\n\"\n",
    "                        current_section = \"\"\n",
    "                        section_type = None\n",
    "                    \n",
    "                    # Append the passage text without tagging\n",
    "                    content += f\"{block.text}\\n\"\n",
    "        \n",
    "        elif isinstance(block, DocxTable):\n",
    "            # Add the current section before the table\n",
    "            if current_section:\n",
    "                content += f\"{current_section}</{section_type}>\\n\"\n",
    "                current_section = \"\"\n",
    "                section_type = None\n",
    "\n",
    "            content += \"<table>\\n\"\n",
    "            for row in block.rows:\n",
    "                row_content = []\n",
    "                for cell in row.cells:\n",
    "                    cell_content = []\n",
    "                    for nested_block in iter_block_items(cell):\n",
    "                        if isinstance(nested_block, Paragraph):\n",
    "                            cell_content.append(nested_block.text)\n",
    "                        elif isinstance(nested_block, DocxTable):\n",
    "                            nested_table_content = parse_nested_table(nested_block)\n",
    "                            cell_content.append(nested_table_content)\n",
    "                    row_content.append(\"|\".join(cell_content))\n",
    "                content += \"|\".join(row_content) + \"\\n\"\n",
    "            content += \"</table>\\n\"\n",
    "\n",
    "    # Add the final section\n",
    "    if current_section:\n",
    "        content += f\"{current_section}</{section_type}>\\n\"\n",
    "\n",
    "    return content\n",
    "\n",
    "def parse_nested_table(table):\n",
    "    nested_table_content = \"<table>\\n\"\n",
    "    for row in table.rows:\n",
    "        row_content = []\n",
    "        for cell in row.cells:\n",
    "            cell_content = []\n",
    "            for nested_block in iter_block_items(cell):\n",
    "                if isinstance(nested_block, Paragraph):\n",
    "                    cell_content.append(nested_block.text)\n",
    "                elif isinstance(nested_block, DocxTable):\n",
    "                    nested_table_content += parse_nested_table(nested_block)\n",
    "            row_content.append(\"|\".join(cell_content))\n",
    "        nested_table_content += \"|\".join(row_content) + \"\\n\"\n",
    "    nested_table_content += \"</table>\"\n",
    "    return nested_table_content\n",
    "\n",
    "\n",
    "\n",
    "def extract_text_from_pptx_s3(pptx_buffer):\n",
    "    \"\"\" Extract Text from pptx files\"\"\"\n",
    "    presentation = Presentation(pptx_buffer)    \n",
    "    text_content = []\n",
    "    for slide in presentation.slides:\n",
    "        slide_text = []\n",
    "        for shape in slide.shapes:\n",
    "            if hasattr(shape, 'text'):\n",
    "                slide_text.append(shape.text)\n",
    "        text_content.append('\\n'.join(slide_text))    \n",
    "    return '\\n\\n'.join(text_content)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4151e645-c9ae-4a62-9a1d-9e58fb67ef2f",
   "metadata": {},
   "source": [
    "The `handle_doc_upload_or_s3 function` is a comprehensive handler for extracting content from various file formats, either from a local file or a file stored in Amazon S3. It takes a file parameter, which can be a local file path or an S3 URI, and returns the extracted content based on the file extension. This handled documents are passed as contet to the LLM.\n",
    "\n",
    "- .pdf, .png, .jpg: It calls the exract_pdf_text_aws function to extract text from the file using AWS Textract or other libraries like PyPDF2 or pytesseract.\n",
    "- .csv: It calls the parse_csv_from_s3 function to parse the CSV file using pandas, detecting the file encoding and sniffing the delimiter.\n",
    "- .xlsx, .xlx: It calls the table_parser_utills function to extract content from Excel files.\n",
    "- .json: It retrieves the file from S3 using the get_s3_obj_from_bucket_ function, reads the file content, and loads it as a JSON object using json.loads.\n",
    "- .txt, .py: It retrieves the file from S3 using the get_s3_obj_from_bucket_ function and reads the file content as plain text.\n",
    "- .docx: It retrieves the file from S3 using the get_s3_obj_from_bucket_ function, reads the file content, creates a BytesIO buffer, and passes it to the extract_text_and_tables function to extract text and tables from the Word document.\n",
    "- .pptx: It retrieves the file from S3 using the get_s3_obj_from_bucket_ function, reads the file content, creates a BytesIO buffer, and passes it to the extract_text_from_pptx_s3 function to extract text from the PowerPoint presentation.\n",
    "- Other file extensions: It uses the textract library to extract content from the file. It retrieves the file from S3 using the get_s3_obj_from_bucket_ function, reads the file content, creates a BytesIO buffer, and passes it to textract.process to extract the text content."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "4c1c2ca3-0874-4da2-94d9-3c02daeeb233",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def handle_doc_upload_or_s3(file):\n",
    "    \"\"\"Handle various document format\"\"\"\n",
    "    dir_name, ext = os.path.splitext(file)\n",
    "    if  ext.lower() in [\".pdf\", \".png\", \".jpg\",\".tif\",\".jpeg\"]:   \n",
    "        content=exract_pdf_text_aws(file)\n",
    "    elif \".csv\"  == ext.lower():\n",
    "        content=parse_csv_from_s3(file)\n",
    "    elif ext.lower() in [\".xlsx\", \".xls\"]:\n",
    "        content=table_parser_utills(file)   \n",
    "    elif  \".json\"==ext.lower():      \n",
    "        obj=get_s3_obj_from_bucket_(file)\n",
    "        content = json.loads(obj['Body'].read())  \n",
    "    elif  ext.lower() in [\".txt\",\".py\",\".md\"]:       \n",
    "        obj=get_s3_obj_from_bucket_(file)\n",
    "        content = obj['Body'].read()\n",
    "    elif \".docx\" == ext.lower():       \n",
    "        obj=get_s3_obj_from_bucket_(file)\n",
    "        content = obj['Body'].read()\n",
    "        docx_buffer = io.BytesIO(content)\n",
    "        content = extract_text_and_tables(docx_buffer)\n",
    "    elif \".pptx\" == ext.lower():       \n",
    "        obj=get_s3_obj_from_bucket_(file)\n",
    "        content = obj['Body'].read()\n",
    "        docx_buffer = io.BytesIO(content)        \n",
    "        content = extract_text_from_pptx_s3(docx_buffer)\n",
    "    else:            \n",
    "        obj=get_s3_obj_from_bucket_(file)\n",
    "        content = obj['Body'].read()\n",
    "        doc_buffer = io.BytesIO(doc_content)\n",
    "        content = textract.process(doc_buffer).decode()\n",
    "    # Implement any other file extension logic \n",
    "    return content"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1cc01092-7b04-4613-951d-67e4e6d32d7a",
   "metadata": {},
   "source": [
    "Stores long-term chat history in DynamoDB or Local Disk\n",
    "\n",
    "   1. This `put_db(messages)`function takes a dictionary of messages and stores it in a DynamoDB table. It uses the user ID and session ID as the primary key to identify the item in the table. If an item with the same user ID and session ID already exists in the table, the function retrieves the existing messages and appends the new messages to the list. Finally, it puts the updated chat item back into the DynamoDB table.\n",
    "2. `save_chat_local(file_path, new_data)`: Saves new chat data to a local JSON file, appending it to the existing data. Handles conversion of Decimal objects to floats.\n",
    "3. `load_chat_local(file_path)`: Loads chat history from a local JSON file stored locally, returning an empty list if the file doesn't exist.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "bc7fdc72-92da-4158-8cbe-b8220a8cd431",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def put_db(messages):\n",
    "    \"\"\"Store long term chat history in DynamoDB\"\"\"    \n",
    "    chat_item = {\n",
    "        \"UserId\": DYNAMODB_USER, # user id\n",
    "        \"SessionId\": SESSIONID, # User session id\n",
    "        \"messages\": [messages],  # 'messages' is a list of dictionaries\n",
    "        \"time\":messages['time']\n",
    "    }\n",
    "    existing_item = DYNAMODB.Table(DYNAMODB_TABLE).get_item(Key={\"UserId\": DYNAMODB_USER, \"SessionId\":SESSIONID})\n",
    "    if \"Item\" in existing_item:\n",
    "        existing_messages = existing_item[\"Item\"][\"messages\"]\n",
    "        chat_item[\"messages\"] = existing_messages + [messages]\n",
    "    response = DYNAMODB.Table(DYNAMODB_TABLE).put_item(\n",
    "        Item=chat_item\n",
    "    )    \n",
    "def save_chat_local(file_path, new_data):\n",
    "    \"\"\"Store long term chat history Local Disk\"\"\"   \n",
    "    try:\n",
    "        # Read the existing JSON data from the file\n",
    "        with open(file_path, \"r\",encoding='utf-8') as file:\n",
    "            existing_data = json.load(file)\n",
    "        if SESSIONID not in existing_data:\n",
    "            existing_data[SESSIONID]=[]\n",
    "    except FileNotFoundError:\n",
    "        # If the file doesn't exist, initialize an empty list\n",
    "        existing_data = {SESSIONID:[]}\n",
    "    # Append the new data to the existing list\n",
    "    from decimal import Decimal\n",
    "    data = [{k: float(v) if isinstance(v, Decimal) else v for k, v in item.items()} for item in new_data]\n",
    "    existing_data[SESSIONID].extend(data)\n",
    "    # Write the updated list back to the JSON file\n",
    "    with open(file_path, \"w\") as file:\n",
    "        json.dump(existing_data, file)\n",
    "        \n",
    "def load_chat_local(file_path):\n",
    "    \"\"\"Load long term chat history from Local\"\"\"   \n",
    "    try:\n",
    "        # Read the existing JSON data from the file\n",
    "        with open(file_path, \"r\",encoding='utf-8') as file:\n",
    "            existing_data = json.load(file)\n",
    "            if SESSIONID in existing_data:\n",
    "                existing_data=existing_data[SESSIONID]\n",
    "            else:\n",
    "                existing_data=[]\n",
    "    except FileNotFoundError:\n",
    "        # If the file doesn't exist, initialize an empty list\n",
    "        existing_data = []\n",
    "    return existing_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41f0c626-b5ca-4352-ab15-3aa93438aab0",
   "metadata": {},
   "source": [
    "Prepares chat history retrieved from either DynamoDB or Local Disk for Claude converation.\n",
    "\n",
    "This function retrieves the chat history from DynamoDB based on the provided `chat_histories` and `cutoff` parameters. It processes the chat history and prepares it for the conversation based on the `claude3` flag and the `LOAD_DOC_IN_ALL_CHAT_CONVO` configuration.\n",
    "   \n",
    "If `claude3` flag, images are processed with claude3 image processor else Amazon Textract or python Libs is used. if `LOAD_DOC_IN_ALL_CHAT_CONVO` all documents in that conversation history is processed and contents are loaded as context in the chat history. The `cutoff` determines the amount of recent chat turns to load into the current conversation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "65c42fff-d32e-4454-94ac-53fc36508da1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def get_chat_history_db(chat_histories, cutoff,claude3):\n",
    "    current_chat=[]\n",
    "    if DYNAMODB_TABLE:\n",
    "        chat_hist=chat_histories['Item']['messages'][-cutoff:] \n",
    "    else:\n",
    "        chat_hist=chat_histories[-cutoff:] \n",
    "    for d in chat_hist:\n",
    "        if d['image'] and claude3 and LOAD_DOC_IN_ALL_CHAT_CONVO:\n",
    "            content=[]\n",
    "            for img in d['image']:\n",
    "                s3 = boto3.client('s3',region_name=REGION)\n",
    "                match = re.match(\"s3://(.+?)/(.+)\", img)\n",
    "                image_name=os.path.basename(img)\n",
    "                _,ext=os.path.splitext(image_name)\n",
    "                if \"jpg\" in ext: ext=\".jpeg\"                        \n",
    "                if match:\n",
    "                    bucket_name = match.group(1)\n",
    "                    key = match.group(2)    \n",
    "                    obj = s3.get_object(Bucket=bucket_name, Key=key)\n",
    "                    base_64_encoded_data = base64.b64encode(obj['Body'].read())\n",
    "                    base64_string = base_64_encoded_data.decode('utf-8')                        \n",
    "                content.extend([{\"type\":\"text\",\"text\":image_name},{\n",
    "                  \"type\": \"image\",\n",
    "                  \"source\": {\n",
    "                    \"type\": \"base64\",\n",
    "                    \"media_type\": f\"image/{ext.lower().replace('.','')}\",\n",
    "                    \"data\": base64_string\n",
    "                  }\n",
    "                }])\n",
    "            content.extend([{\"type\":\"text\",\"text\":d['user']}])\n",
    "            current_chat.append({'role': 'user', 'content': content})\n",
    "        elif d['document'] and LOAD_DOC_IN_ALL_CHAT_CONVO:\n",
    "            doc='Here are the documents:\\n'\n",
    "            for docs in d['document']:\n",
    "                uploads=handle_doc_upload_or_s3(docs)\n",
    "                doc_name=os.path.basename(docs)\n",
    "                doc+=f\"<{doc_name}>\\n{uploads}\\n</{doc_name}>\\n\"\n",
    "            if not claude3 and d[\"image\"]:\n",
    "                for docs in d['image']:\n",
    "                    uploads=handle_doc_upload_or_s3(docs)\n",
    "                    doc_name=os.path.basename(docs)\n",
    "                    doc+=f\"<{doc_name}>\\n{uploads}\\n</{doc_name}>\\n\"\n",
    "            current_chat.append({'role': 'user', 'content': [{\"type\":\"text\",\"text\":doc+d['user']}]})\n",
    "        else:\n",
    "            current_chat.append({'role': 'user', 'content': [{\"type\":\"text\",\"text\":d['user']}]})\n",
    "        current_chat.append({'role': 'assistant', 'content': d['assistant']})  \n",
    "    return current_chat, chat_hist"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5abbf370-d81f-4a8e-b9df-cfca4ba311e5",
   "metadata": {},
   "source": [
    "Processes the streamed response from the Bedrock model and extracts the generated text.\n",
    "\n",
    "Invokes the Bedrock Claude model with the provided chat history, system message, prompt, and optional image(s)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "f2bbb611-737c-4915-8dc3-15925091cca5",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def bedrock_streemer(response):\n",
    "    stream = response.get('body')\n",
    "    answer = \"\"\n",
    "    i = 1\n",
    "    if stream:\n",
    "        for event in stream:\n",
    "            chunk = event.get('chunk')\n",
    "            if  chunk:\n",
    "                chunk_obj = json.loads(chunk.get('bytes').decode())\n",
    "                if \"delta\" in chunk_obj:                    \n",
    "                    delta = chunk_obj['delta']\n",
    "                    if \"text\" in delta:\n",
    "                        text=delta['text'] \n",
    "                        print(text, end=\"\")\n",
    "                        answer+=str(text)       \n",
    "                        i+=1\n",
    "                if \"amazon-bedrock-invocationMetrics\" in chunk_obj:\n",
    "                    input_tokens= chunk_obj['amazon-bedrock-invocationMetrics']['inputTokenCount']\n",
    "                    output_tokens=chunk_obj['amazon-bedrock-invocationMetrics']['outputTokenCount']\n",
    "                    print(f\"\\nInput Tokens: {input_tokens}\\nOutput Tokens: {output_tokens}\")\n",
    "    return answer,input_tokens, output_tokens\n",
    "\n",
    "def bedrock_claude_(chat_history,system_message, prompt,model_id,image_path=None):\n",
    "\n",
    "    content=[]\n",
    "    if image_path:       \n",
    "        if not isinstance(image_path, list):\n",
    "            image_path=[image_path]      \n",
    "        for img in image_path:\n",
    "            s3 = boto3.client('s3',region_name=REGION)\n",
    "            match = re.match(\"s3://(.+?)/(.+)\", img)\n",
    "            image_name=os.path.basename(img)\n",
    "            _,ext=os.path.splitext(image_name)\n",
    "            if \"jpg\" in ext: ext=\".jpeg\"                        \n",
    "            if match:\n",
    "                bucket_name = match.group(1)\n",
    "                key = match.group(2)    \n",
    "                obj = s3.get_object(Bucket=bucket_name, Key=key)\n",
    "                base_64_encoded_data = base64.b64encode(obj['Body'].read())\n",
    "                base64_string = base_64_encoded_data.decode('utf-8')\n",
    "            content.extend([{\"type\":\"text\",\"text\":image_name},{\n",
    "              \"type\": \"image\",\n",
    "              \"source\": {\n",
    "                \"type\": \"base64\",\n",
    "                \"media_type\": f\"image/{ext.lower().replace('.','')}\",\n",
    "                \"data\": base64_string\n",
    "              }\n",
    "            }])\n",
    "\n",
    "    content.append({\n",
    "        \"type\": \"text\",\n",
    "        \"text\": prompt\n",
    "            })\n",
    "    chat_history.append({\"role\": \"user\",\n",
    "            \"content\": content})\n",
    "    prompt = {\n",
    "        \"anthropic_version\": \"bedrock-2023-05-31\",\n",
    "        \"max_tokens\": 1500,\n",
    "        \"temperature\": 0.5,\n",
    "        \"system\":system_message,\n",
    "        \"messages\": chat_history\n",
    "    }\n",
    "    \n",
    "    prompt = json.dumps(prompt)\n",
    "    response = bedrock_runtime.invoke_model_with_response_stream(body=prompt, modelId=model_id, accept=\"application/json\", contentType=\"application/json\")\n",
    "    answer,input_tokens,output_tokens=bedrock_streemer(response) \n",
    "    return answer, input_tokens, output_tokens"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68d250fd-ae18-40e3-b799-6b08c905a435",
   "metadata": {},
   "source": [
    "Invokes the Bedrock Claude model with retries and exponential backoff in case of throttling errors.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "e9c6d59a-ae7e-4b73-93cd-eb385315c476",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def _invoke_bedrock_with_retries(current_chat, chat_template, question, model_id, image_path):\n",
    "    max_retries = 5\n",
    "    backoff_base = 2\n",
    "    max_backoff = 3  # Maximum backoff time in seconds\n",
    "    retries = 0\n",
    "\n",
    "    while True:\n",
    "        try:\n",
    "            response,input_tokens,output_tokens= bedrock_claude_(current_chat, chat_template, question, model_id, image_path)\n",
    "            return response,input_tokens,output_tokens\n",
    "        except ClientError as e:\n",
    "            if e.response['Error']['Code'] == 'ThrottlingException':\n",
    "                if retries < max_retries:\n",
    "                    # Throttling, exponential backoff\n",
    "                    sleep_time = min(max_backoff, backoff_base ** retries + random.uniform(0, 1))\n",
    "                    time.sleep(sleep_time)\n",
    "                    retries += 1\n",
    "                else:\n",
    "                    raise e\n",
    "            elif e.response['Error']['Code'] == 'ModelStreamErrorException':\n",
    "                if retries < max_retries:\n",
    "                    # Throttling, exponential backoff\n",
    "                    sleep_time = min(max_backoff, backoff_base ** retries + random.uniform(0, 1))\n",
    "                    time.sleep(sleep_time)\n",
    "                    retries += 1\n",
    "                else:\n",
    "                    raise e\n",
    "            elif e.response['Error']['Code'] == 'EventStreamError':\n",
    "                if retries < max_retries:\n",
    "                    # Throttling, exponential backoff\n",
    "                    sleep_time = min(max_backoff, backoff_base ** retries + random.uniform(0, 1))\n",
    "                    time.sleep(sleep_time)\n",
    "                    retries += 1\n",
    "                else:\n",
    "                    raise e\n",
    "            else:\n",
    "                # Some other API error, rethrow\n",
    "                raise\n",
    "                "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0d1444e-a734-4a80-bce2-f51730af9517",
   "metadata": {},
   "source": [
    "#### Chat Function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ee7ea26-8cb1-471c-867a-fb028125d094",
   "metadata": {},
   "source": [
    "Conducts a conversation with the Bedrock Claude model based on the user's question and optional uploaded documents.\n",
    "\n",
    "   This function takes a user's question and a list of document paths (optional) as input. It retrieves the past chat\n",
    "   history from DynamoDB (if configured) or uses local storage. It prepares the chat template based on whether\n",
    "   documents are provided or not. If documents are provided, it handles the document uploads and extracts the text\n",
    "   content. If the Claude3 model is used, it handles images separately. The function then invokes the Bedrock Claude\n",
    "   model with retries and exponential backoff in case of throttling errors. The conversation history is stored in\n",
    "   DynamoDB (if configured) or local disk for future reference.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "49c188da-5167-4495-b951-f95ea40a59d4",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from typing import List\n",
    "def conversation_bedroc_chat_(question, model_id,upload_doc: List[str]):\n",
    "    \"\"\"\n",
    "    Function takes a user query and a document path (from S3 or Local)\n",
    "    passing a document path is optional\n",
    "    \"\"\"    \n",
    "    num_retries=0\n",
    "    local_chat_file_name = f\"chat-history.json\"\n",
    "    if not isinstance(upload_doc, list):\n",
    "        raise TypeError(\"documents must be in a list format\")\n",
    "        \n",
    "    # Check if Claude3 model is used and handle images with the CLAUDE3 Model\n",
    "    claude3=False\n",
    "    if \"sonnet\" in model_id or \"haiku\" in model_id:\n",
    "        claude3=True\n",
    "    current_chat=[]\n",
    "   \n",
    "    # Retrieve past chat history from Dynamodb\n",
    "    if DYNAMODB_TABLE:\n",
    "        chat_histories = DYNAMODB.Table(DYNAMODB_TABLE).get_item(Key={\"UserId\": DYNAMODB_USER, \"SessionId\":SESSIONID})\n",
    "        if \"Item\" in chat_histories:            \n",
    "            current_chat,chat_hist=get_chat_history_db(chat_histories, CHAT_HISTORY_LENGTH,claude3)\n",
    "        else:\n",
    "            chat_hist=[]\n",
    "    # Retrieve from local\n",
    "    else:\n",
    "        chat_histories=load_chat_local(local_chat_file_name)\n",
    "        if chat_histories:\n",
    "            current_chat,chat_hist=get_chat_history_db(chat_histories, CHAT_HISTORY_LENGTH,claude3)\n",
    "    ## prompt template for when a user uploads a doc\n",
    "    doc_path=[]\n",
    "    image_path=[]\n",
    "    full_doc_path=[]\n",
    "    doc=\"\"\n",
    "    if upload_doc:  \n",
    "        doc='I have provided documents and/or images.\\n'\n",
    "        for ids,docs in enumerate(upload_doc):\n",
    "            _,extensions=os.path.splitext(docs)\n",
    "            if not docs.startswith(\"s3://\"):\n",
    "                docs=put_obj_in_s3_bucket_(docs)\n",
    "            full_doc_path.append(docs)\n",
    "            if extensions in [\".jpg\",\".jpeg\",\".png\",\".gif\",\".webp\"] and claude3:       \n",
    "                image_path.append(docs)                \n",
    "                continue\n",
    "                \n",
    "        new_upload_doc = [item for item in full_doc_path if item not in image_path]\n",
    "        results, errors, result_string=process_files(new_upload_doc)    \n",
    "        if errors:\n",
    "            print(errors)\n",
    "        doc+= result_string\n",
    "        with open(\"prompt/doc_chat.txt\",\"r\",encoding='utf-8') as f:\n",
    "            chat_template=f.read()       \n",
    "    else:        \n",
    "        # Chat template for open ended query\n",
    "        with open(\"prompt/chat.txt\",\"r\",encoding='utf-8') as f:\n",
    "            chat_template=f.read()    \n",
    "    response,input_tokens,output_tokens=_invoke_bedrock_with_retries(current_chat, chat_template, doc+question, model_id, image_path)\n",
    "    chat_history={\"user\":question,\n",
    "    \"assistant\":response,\n",
    "    \"image\":image_path,\n",
    "    \"document\":new_upload_doc if upload_doc else [],\n",
    "    \"modelID\":model_id,\n",
    "    \"time\":str(time.time()),\n",
    "    \"input_token\":round(input_tokens) ,\n",
    "    \"output_token\":round(output_tokens)}         \n",
    "                 \n",
    "    #store convsation memory in DynamoDB table\n",
    "    if DYNAMODB_TABLE:\n",
    "        put_db(chat_history)\n",
    "    # use local disk for storage\n",
    "    else:        \n",
    "        save_chat_local(local_chat_file_name,[chat_history])\n",
    "    return response,doc"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "75e31287-db05-4e8a-86bc-c46ce91da664",
   "metadata": {},
   "source": [
    "#### Query the the chat bot with your questions.\n",
    "Also takes a document path(s) stored in s3 or local. Once a documents path is passed, a different prompt template is triggered."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "id": "f1e2f81b-4c73-4de1-b4c8-2c051914e3f1",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A solar eclipse occurs when the Moon passes between the Sun and the Earth, temporarily obscuring the Sun's light from reaching parts of the Earth. There are two types of solar eclipses:\n",
      "\n",
      "1. **Total Solar Eclipse**: This happens when the Moon's apparent diameter in the sky is larger than the Sun's, blocking all direct sunlight and casting a moon's umbral shadow on parts of Earth's surface. During a total solar eclipse, the Sun's corona (outer atmosphere) becomes visible, creating a spectacular sight.\n",
      "\n",
      "2. **Partial Solar Eclipse**: This occurs when the Moon only partially obscures the Sun's disk, and a portion of the Sun remains visible from Earth.\n",
      "\n",
      "The frequency of solar eclipses depends on the type:\n",
      "\n",
      "- **Total Solar Eclipses**: These are relatively rare, occurring somewhere on Earth's surface approximately once every 18 months. However, at any given location on Earth, a total solar eclipse will only be visible on average once every 375 years.\n",
      "\n",
      "- **Partial Solar Eclipses**: These are more common, occurring at least twice a year and up to five times a year when the Moon's ascending and descending nodes align with the Sun's path in the sky.\n",
      "\n",
      "The timing and location of solar eclipses can be precisely calculated and predicted years in advance due to the regular motions of the Earth, Moon, and Sun. The next total solar eclipse visible from parts of the United States will occur on April 8, 2024, and will be known as the \"Great American Eclipse.\"\n",
      "Input Tokens: 53\n",
      "Output Tokens: 328\n"
     ]
    }
   ],
   "source": [
    "question=\"\"\"Explain the solar eclipse and how often does it happen?\"\"\"\n",
    "model_id=\"anthropic.claude-3-sonnet-20240229-v1:0\"#\"anthropic.claude-3-sonnet-20240229-v1:0\"\"anthropic.claude-v2\",\"anthropic.claude-3-haiku-20240307-v1:0\"\n",
    "docu=[]  # pass a list of document names (strings) in local storage or S3 else leave as an empty list\n",
    "res,d=conversation_bedroc_chat_(question, model_id,docu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f837a3a2-8252-466d-b058-67ca50669f27",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 57,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.trn1.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 58,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1.32xlarge",
    "vcpuNum": 128
   },
   {
    "_defaultOrder": 59,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.trn1n.32xlarge",
    "vcpuNum": 128
   }
  ],
  "instance_type": "ml.t3.xlarge",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}