Azure OpenAI Terraform deployment for sample chatbot

This sample application deploys an AI-powered document search using Azure OpenAI Service, Azure Kubernetes Service (AKS), and a Python application leveraging the Llama index ans Streamlit. The application will be deployed within a virtual network to ensure security and isolation. Users will be able to upload documents and ask questions based on the content of the uploaded documents.

Prerequisites

• Azure subscription. If you don't have an Azure subscription, create a free account before you begin.
• Subscription access to Azure OpenAI service. Request Access to Azure OpenAI Service here.
• Terraform.

The easiest way to run this sample is to run it creating a new GitHub Codespace

Quickstart

(Optional) configure GitHub Codespaces secrets to access your Azure subscription

• Run the following command to create a service principal with the "Owner" role for a specific subscription, and outputs its information in JSON format.

  az ad sp create-for-rbac --role="Owner" --scopes="/subscriptions/<SUBSCRIPTION_ID>" -o json

• In your github account go to Codespaces and Create a new Codespace with "Azure-Sample/azure-openai-terraform-deployment-sample" repository and select the main branch.

  

• In your github account, go to Settings. On the left pane, select Codespaces tab and create a secret for ARM_CLIENT_ID, ARM_CLIENT_SECRET, ARM_SUBSCRIPTION_ID and ARM_TENANT_ID values, as shown in the image below. For each secret, on the Repository access section, click on the "Select repositories" dropdown menu and select "Azure-Samples/azure-openai-terraform-deployment-sample".

  

• Follow this link to create a new GitHub Codespace.

Run the Terraform

• Clone or fork this repository. (Skip if using GitHub codespaces)

  git clone https://github.com/Azure-Samples/azure-openai-terraform-deployment-sample/
  cd azure-openai-terraform-deployment-sample
  

• Go to the infra folder and run the following command to initialize your working directory.

  cd infra
  terraform init

• Run terraform apply to deploy all the necessary resources on Azure.

  terraform apply

• Run the following command. This script retrieves the AKS cluster credentials, logs in to the ACR, builds and pushes a Docker image, creates a federated identity, and deploys resources to the Kubernetes cluster using a YAML manifest.

  terraform output -raw installation-script | bash

• Get the external ip address of the service by running the command bellow.

  kubectl get services -n chatbot

• Copy the external ip address and paste it in your browser. The application should load in a few seconds.

• You can test the application by uploading a sample file.txt with the following content:

  Saverio and Sofia both work at Microsoft.
  They live in Switzerland.
  Saverio is Italian.
  Sofia is from Portugal.
  

  And then ask a question like the following screenshot:

  

• Access the Grafana dashboard by running the following command.

  kubectl port-forward svc/grafana 8080:80

  And point your browser to http://localhost:8080.

  You can browse to the metrics explorer and see the traces collected by the application. In the traces you are able to inspect the actual prompt that the llama_index library is using to query the OpenAI service.

  

Resources

• Azure OpenAI
• Azure OpenAI Terraform verified module.