All About Lambda Response Streaming

This repo consists of 3 use cases for response streaming

1. Streaming data from DynamoDB.
2. Streaming data from ChatGPT and comparing it to a regular API response.
3. Streaming data from Bedrock on a custom python lambda runtime and comparing it to ChatGPT Streaming.

Pre Requisites

• AWS SAM CLI
• ShelbySAM - npm i -g @antstackio/shelbysam
• Docker
• Python3
• You need to add openapi secrets in AWS secrete manager service:
  • CHAT_GPT as secret name.
  • chat gpt api key as API_KEY secret.
  • chat gpt org id as ORG_ID secret.

Initialization

samconfig.toml

The path to the file is /backend/samconfig.toml. This file contains the stack name, the s3 prefix, and the region of deployment. Note that Amazon Bedrock is supported in only 5 regions, so choose your region carefully

parameters

The path to parameter file is /backend/infra_resources/Parameters.yaml. Change the parameter values as required.

Deployment

Backend

1. Navigate to the backend folder.
2. Run shelbysam construct. After making changes to the paramters and other infra resources, run this command to generate the template.yaml file.
3. Run sam build. Note: When running this command docker must be running as well. The first execution will take a long time as the docker build time is quite large.
4. Run sam deploy --resolve-image-repos. The additional parameter resolve-image-repos to sam deploy will automatically fetch the docker images for the lambda.

Backend Post Deployment

1. Navigate to the root of the repository
2. Modify the scan_db_table and query_db_table variable to match the deployment. You can fetch this from the cloudformation output as well.
3. Run pip3 install -r requirements.txt to install the required packages.
4. Run python3 uploadToDynamoDb.py. This will update the base data required for the DynamoDB streaming use case.

Frontend

1. cd frontend. Navigate to the frontend directory.
2. npm install. Install the required packages.
3. npm run build. Build the frontend code.
4. aws s3 sync dist s3://{BucketName}. Replace BucketName with the bucket name used for deployment. It is listed as HostedBucketName in the output section of the cloudformation.

The cloudfront url of the deployed application is available in the output section as CloudFrontDomainName. Once the above mentioned steps are completed, you can access the link to use the application.