Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added new feature to integrate azure services using managed identities #442

Open
wants to merge 18 commits into
base: main
Choose a base branch
from

Conversation

FreddyAyala
Copy link

@FreddyAyala FreddyAyala commented Oct 6, 2024

This pull request introduces:

  • support for Managed Identities, enhancing security and simplifying secret management. Key changes include updates to documentation, infrastructure templates, and backend TS classes.
  • add_localdev_roles.ps1 helper script - add the required RBAC roles to the locally logged-in user to enable local dev.
  • appreg_setup.ps1 - automated creation of the App Registration in Entra and updates the web app too.
  • Debug logging: set the DEBUG=true environment variable and you'll get extensive logging from the app (warning, may contain secrets/private info)

Documentation Updates:

  • Added a new section on using Managed Identities for the Azure Chat Solution Accelerator, detailing security advantages, services using Managed Identities, and deployment instructions: docs/10.managed-identities.md
  • Added "What's New" to the main README.md

Infrastructure Updates:

  • Introduced a new parameter disableLocalAuth in infra/main.bicep to toggle authentication by key, enforcing RBAC using Managed Identities. (infra/main.bicep) [1] [2]
  • Updated infra/main.json to include the disableLocalAuth parameter and its usage across various Azure services configurations. (infra/main.json) [1] [2] [3] [4] [5] [6] [7] [8] [9] [10]

Deployment Configuration:

  • The bicep files now output the majority of the environment variables required for the app to run -- they are stored in the azd environment (/.azure/<env_name>/.env) this enables developers (or Copilot) to copy most of the values directly into .env.local
  • Modified the deployment instructions to ensure the parameter disableLocalAuth is set to true for using Managed Identities and updated environment variables accordingly. (infra/main.json) [1] [2]

These changes collectively enhance the security posture of the Azure Chat deployment by leveraging Managed Identities, while also simplifying secret management and access control.

Code changes:

  • modified various backend classes to use DefaultAzureCredential
  • Added debug logging statements and DEBUG environment variable

New Scripts:

  • add_localdev_roles.ps1 - adds the required RBAC roles to the locally logged-in user to enable local dev
  • appreg_setup.ps1 - automated creation of the App Registration

@FreddyAyala
Copy link
Author

FreddyAyala commented Oct 6, 2024

Hey there @thivy @davidxw ,
I've spent the last few days adding a new feature that enables the use of managed identities with the accelerator, except for Azure Speech, which I couldn't get to work reliably with managed identities and TypeScript.
As you might know, the FSI initiative is locking down tenants and enforcing the use of managed identities for internal tenants, particularly for CosmosDB. This change broke our solution, so I took the time to modify the infrastructure code and application services to support managed identities. This enhancement allows us to eliminate the risks associated with key sharing and deploy the solution in locked-down tenants.
Please take a look when you have a chance. I've conducted extensive testing to ensure everything works correctly.

@pyrox82
Copy link

pyrox82 commented Nov 12, 2024

Hi, any updates on this? I would really like to use this solution with managed identities.

Thank you in advance.

@davidxw davidxw requested a review from Copilot December 9, 2024 22:55

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 6 out of 13 changed files in this pull request and generated no suggestions.

Files not reviewed (7)
  • infra/main.bicep: Language not supported
  • infra/main.json: Language not supported
  • infra/resources.bicep: Language not supported
  • src/package.json: Language not supported
  • src/features/common/services/document-intelligence.ts: Evaluated as low risk
  • src/features/common/services/openai.ts: Evaluated as low risk
  • src/features/chat-page/chat-services/chat-document-service.ts: Evaluated as low risk
Comments skipped due to low confidence (2)

src/features/common/services/ai-search.ts:24

  • Rename function to follow camelCase convention: getCredential.
export const GetCredential = () => {

src/features/common/services/azure-storage.ts:20

  • [nitpick] The error message should specify the exact environment variable name, e.g., "AZURE_STORAGE_ACCOUNT_NAME or AZURE_STORAGE_ACCOUNT_KEY environment variable is not set".
throw new Error("Azure Storage Account not configured correctly, check environment variables.");
@leongj leongj self-assigned this Dec 20, 2024
@leongj
Copy link
Collaborator

leongj commented Dec 26, 2024

Hey @FreddyAyala thanks so much for the work you've put into this PR, it looks great.
Let's change a few things if that's ok:

  1. debug logging - connectivity logging is good, but let's remove any logging that contains actual data (e.g. the AI search queries and responses) or credentials (please check some of the connection logging).
  2. local user MI - to support local dev -- can we please add a script that assigns the local user (e.g. az login) with the required IAM roles -- that will make local dev easier. Happy if we want to call this script directly from azd
  3. other stuff - a few other things that I made comments on directly above

@@ -889,7 +940,8 @@
"kind": "OpenAI",
"properties": {
"customSubDomainName": "[variables('openai_dalle_name')]",
"publicNetworkAccess": "Enabled"
"publicNetworkAccess": "Enabled",

Check failure

Code scanning / templateanalyzer

Restrict Cognitive Service endpoints. Error

By default, public network access is enabled for a Cognitive Service account. Service Endpoints and Private Link can be leveraged to restrict access to PaaS endpoints. When access is restricted, access by malicious actor is from an unauthorized virtual network is mitigated.
Configure service endpoints and private links where appropriate.
@@ -889,7 +940,8 @@
"kind": "OpenAI",
"properties": {
"customSubDomainName": "[variables('openai_dalle_name')]",
"publicNetworkAccess": "Enabled"
"publicNetworkAccess": "Enabled",

Check failure

Code scanning / templateanalyzer

Use Cognitive Service Private Endpoints. Error

By default, a public endpoint is enabled for Cognitive Services accounts. The public endpoint is used for all access except for requests that use a Private Endpoint. Access through the public endpoint can be disabled or restricted to authorized virtual networks.
Data exfiltration is an attack where an malicious actor does an unauthorized data transfer. Private Endpoints help prevent data exfiltration by an internal or external malicious actor. They do this by providing clear separation between public and private endpoints. As a result, broad access to public endpoints which could be operated by a malicious actor are not required.
@@ -889,7 +940,8 @@
"kind": "OpenAI",
"properties": {
"customSubDomainName": "[variables('openai_dalle_name')]",
"publicNetworkAccess": "Enabled"
"publicNetworkAccess": "Enabled",
"disableLocalAuth": "[parameters('disableLocalAuth')]"

Check failure

Code scanning / templateanalyzer

Use identity-based authentication for Cognitive Services accounts. Error

To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits.
With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing.
Once you decide to use Azure AD authentication, you can disable authentication using keys.
@@ -917,7 +969,169 @@
"location": "[parameters('location')]",
"tags": "[parameters('tags')]",
"kind": "StorageV2",
"sku": "[parameters('storageServiceSku')]"
"sku": "[parameters('storageServiceSku')]",
"properties": {

Check failure

Code scanning / templateanalyzer

Configure Azure Storage firewall. Error

By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action.
After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: Azure services on the trusted service list. IP address or CIDR range. Private endpoint connections. Azure virtual network subnets with a Service Endpoint.
@@ -917,7 +969,169 @@
"location": "[parameters('location')]",
"tags": "[parameters('tags')]",
"kind": "StorageV2",
"sku": "[parameters('storageServiceSku')]"
"sku": "[parameters('storageServiceSku')]",
"properties": {

Check failure

Code scanning / templateanalyzer

Storage Account minimum TLS version. Error

The minimum version of TLS that Azure Storage Accounts accept for blob storage is configurable. Older TLS versions are no longer considered secure by industry standards, such as PCI DSS.
Storage Accounts lets you disable outdated protocols and enforce TLS 1.2. By default, TLS 1.0, TLS 1.1, and TLS 1.2 is accepted.
@@ -917,7 +969,169 @@
"location": "[parameters('location')]",
"tags": "[parameters('tags')]",
"kind": "StorageV2",
"sku": "[parameters('storageServiceSku')]"
"sku": "[parameters('storageServiceSku')]",
"properties": {

Check failure

Code scanning / templateanalyzer

Disallow anonymous access to blob service. Error

Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted.
Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously.
@@ -463,6 +474,7 @@
properties: {
customSubDomainName: openai_dalle_name
publicNetworkAccess: 'Enabled'
disableLocalAuth: disableLocalAuth

Check failure

Code scanning / templateanalyzer

Use identity-based authentication for Cognitive Services accounts. Error

To send requests to Cognitive Services endpoints, each request must include an authentication header. Cognitive Services endpoints supports authentication with keys or tokens. Using an Azure AD token instead of a cryptographic key has some additional security benefits.
With Azure AD authentication, the identity is validated against Azure AD identity provider. Using Azure AD identities centralizes identity management and auditing.
Once you decide to use Azure AD authentication, you can disable authentication using keys.
@@ -506,6 +519,9 @@
tags: tags
kind: 'StorageV2'
sku: storageServiceSku
properties:{

Check failure

Code scanning / templateanalyzer

Configure Azure Storage firewall. Error

By default, storage accounts accept connections from clients on any network. To limit access to selected networks, you must first change the default action.
After changing the default action from Allow to Deny, configure one or more rules to allow traffic. Traffic can be allowed from: Azure services on the trusted service list. IP address or CIDR range. Private endpoint connections. Azure virtual network subnets with a Service Endpoint.
@@ -506,6 +519,9 @@
tags: tags
kind: 'StorageV2'
sku: storageServiceSku
properties:{

Check failure

Code scanning / templateanalyzer

Storage Account minimum TLS version. Error

The minimum version of TLS that Azure Storage Accounts accept for blob storage is configurable. Older TLS versions are no longer considered secure by industry standards, such as PCI DSS.
Storage Accounts lets you disable outdated protocols and enforce TLS 1.2. By default, TLS 1.0, TLS 1.1, and TLS 1.2 is accepted.
@@ -506,6 +519,9 @@
tags: tags
kind: 'StorageV2'
sku: storageServiceSku
properties:{

Check failure

Code scanning / templateanalyzer

Disallow anonymous access to blob service. Error

Blob containers in Azure Storage Accounts can be configured for private or anonymous public access. By default, containers are private and only accessible with a credential or access token. When a container is configured with an access type other than private, anonymous access is permitted.
Anonymous access to blobs or containers can be restricted by setting allowBlobPublicAccess to false. This enhanced security setting for a storage account overrides the individual settings for blob containers. When you disallow public access for a storage account, blobs are no longer accessible anonymously.
@leongj
Copy link
Collaborator

leongj commented Jan 17, 2025

@thivy @davidxw @FreddyAyala this PR is now ready for your review and testing! Thanks so much Freddy for the original work you did on this one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants