Azure OpenAI On Your Data

The following is a custom copilot that uses the Azure OpenAI Chat Completions API Azure OpenAI On Your Data feature to facilitate RAG (retrieval augmented generation). You can chat with your data in Azure AI Search, Azure Blob Storage, URL/web address, Azure Cosmos DB for MongoDB vCore, uploaded files, and Elasticsearch.

• Azure OpenAI On Your Data
  • Summary
  • Prerequisites
  • Setting up your custom copilot in Azure OpenAI Studio
  • Setting up and testing your custom copilot in Visual Studio Code
  • Provisioning, Deploying, and Publishing your custom copilot
    • Assigning Cognitive Service OpenAI User role to your deployed App Service resource
  • Supplementary Details and Tips
    • Enable your Custom Copilot for Group Chats and Channels
    • Enable Out of Scope Conversations

Summary

You can deploy to a standalone Teams app (preview) directly from Azure OpenAI Studio, enabling you to bring conversational experience to your users in Teams to improve operational efficiency and democratize access of information. This Teams app is configured to users within a single tenant and personal chat (non-group chat) scenarios. See the Enable your Custom Copilot for Group Chats and Channels section to enable group chat scenarios noting that the AI response quality from Azure OpenAI On Your Data has not been fully tested for group chats.

This guide will show you have the set up your custom copilot for Teams using Azure OpenAI Studio and Teams Toolkit.

Prerequisites

Install	For using...
Visual Studio Code	Typescript build environments. Use the latest version.
Teams Toolkit (5.3.x or greater)	Microsoft Visual Studio Code extension that creates a project scaffolding for your app. Use the latest version.
Node.js (16 or 18)	Back-end JavaScript runtime environment. Recommended to use Node Version 16.x or 18.x, Node version >=19 is not supported. For more information, see Node.js version compatibility table for project type
Microsoft Teams	Access to a Microsoft Teams account with the appropriate permissions to install an app, enable custom Teams apps, and turn on custom app uploading.
Azure OpenAI	Deploy Azure OpenAI large language models and test your custom copilot ideas in the Azure OpenAI Studio Playground. If you want to host your app or access resources in Microsoft Azure, you must create an Azure OpenAI service.
Azure CLI	The Azure Command-Line Interface (CLI) is a cross-platform command-line tool to connect to Azure and execute administrative commands on Azure resources. For more information on setting up environment variables, see the Azure SDK documentation.
Cognitive Services OpenAI User Role	Your Azure account has been assigned “Cognitive Services OpenAI user” or “Cognitive Services OpenAI Contributor” role of the Azure OpenAI resource you are using allowing your account to make Azure OpenAI API calls. For more information see Using your data with Azure OpenAI securely and Add role assignment to an Azure OpenAI resource for instructions on setting this role in the Azure portal.

Setting up your custom copilot in Azure OpenAI Studio

1. Follow the use your data quickstart instructions to add your data using Azure OpenAI Studio chat playground.

2. After adding your data, click Deploy to and then A new Teams app(preview).

3. Enter the name of your Teams app.

4. Click download to download your Teams app as a zip file.

5. Open the location of where you downloaded the zip file and extract the zip file.

Setting up and testing your custom copilot in Visual Studio Code

Note: Testing this sample requires that you are logged into Azure CLI and you have Cognitive Services OpenAI User role assigned to you per the prerequisites.

1. Go to Visual Studio Code.

2. Select File > Open Folder.

3. Go to the location where you extracted your Teams app folder and select it.

4. If you chose API key in data connection, manually copy and paste your Azure AI Search key in src\prompts\chat\config.json file. Your Azure AI Search Key can be found in Azure OpenAI Studio Playground by clicking the View code button and looking under Azure Search Resource Key. If you chose system assigned managed identity, you can skip this step. Learn more about different data connection options in the Data connection section .

5. Open the Visual Studio terminal by selecting View > Terminal and log into Azure CLI selecting the Azure account that you assigned Cognitive Service OpenAI User role to. Use the following command to log in:

     az login

6. To debug your app, press the F5 key or from the left pane, select RUN AND DEBUG ▷ (Ctrl+Shift+D) and then select Debug in (Test tool) from the dropdown list. Select Run > Start Debugging (F5). The Test Tool opens in a webpage where you can chat with your custom copilot.

Provisioning, Deploying, and Publishing your custom Copilot

After you've tested your copilot, you can provision, deploy, and publish your Teams app by selecting the Teams Toolkit Extension on the left pane in Visual Studio. Run the separate provision, deploy, and publish stages in Lifecycle section. You may be asked to sign into your Microsoft 365 account where you have permissions to upload custom apps and your Azure Account.

IMPORTANT As this sample uses managed identity, for your custom copilot to generate responses you must assign Cognitive Service OpenAI User role to your custom copilot’s App Service resource group after deploying your app to Azure.

1. Provision your app
2. Deploy to Azure
3. Publish to Teams

Assigning Cognitive Service OpenAI User role to your deployed App Service resource

As this sample uses managed identity, you must enable assign Cognitive Service OpenAI User role to your custom copilot’s App Service resource group after deploying your app to Azure in order for your deployed custom copilot to receive responses from Azure OpenAI.

1. Go to Azure portal and select resource groups

2. Select the resource group you deployed your custom copilot to

3. Select the App Service resource

4. Go to settings > identity and enable system assigned identity by selecting On

5. Select Save to enable system assigned identity.

6. Click Azure role assignments

7. Click add role assignments.

8. Under Scope select Resource group

9. Under Subscription select the Azure subscription of your Azure OpenAI resource

10. Under Resource group select your Azure OpenAI resource

11. Under Role select Cognitive Service OpenAI user

12. Select save to finish assigning the role

Now, your published custom copilot Teams app is ready for use.

Supplementary Details and Tips

Enable your Custom Copilot for Group Chats and Channels

This custom copilot sample is pre-configured for only personal chats (1 on 1) during preview due to ongoing testing from Azure OpenAI On Your Data to determine the effects of group chats on AI response quality. Group chats can be enabled with the understanding that the AI response quality has not been fully tested for these scenarios.

A custom copilot can be mentioned ("@customcopilotname") in a channel if it has been added to the team. Note that additional replies to a custom copilot in a channel require @ mentioning the custom copilot. It will not respond to replies where it isn't mentioned. See here for more information.

To enable group chats:

1. Go to appPackage\manifest.json file

2. Add the team and groupchat parameters to your bots' scopes in additional to the existing personal scope.

    "bots": [
            {
                "botId": "${{BOT_ID}}",
                "scopes": [
                    "personal",
                    "team",
                    "groupchat"
                ],
   

Enable Out of Scope Conversations

You can modify the settings in the Data parameters section in src\prompts\chat\config.json file. The in_scope parameter configures the chatbot's approach to handling queries unrelated to the data source or when search documents are insufficient for a complete answer. When this setting is set to false, the model supplements its responses with its own knowledge in addition to your documents.

By default, the in_scope parameter is set to true resulting in the model attempting to only rely on your documents for responses. Out of scope questions may receive the following response:

“The requested information is not available in the retrieved data. Please try another query or topic.”

For more information please see Runtime parameters