Skip to content

Commit

Permalink
Add Azure Key Vault Extension (#218)
Browse files Browse the repository at this point in the history
* Initialize file structure for keyvault extension.

* use quarkus cli application for testing.

* add internal extension for azure identity

* add dynamic-proxy.

* add reflectiveHierarchyClass

* complete integration test.

* remove unused dependency.

* rename test sample.

* procude SecretAsyncClient.

* add ut.

* complete integration test

* create doc for key vault extension.

* add ci for key vault extension.

* code clean up

* clean up import

* update integration test.

* update top README.

* Update README of integration-test for keyvault

* add KeyVaultSecretResourceIT.

* fix com.microsoft.aad.msal4j.AbstractClientApplicationBase$Builder not found error.

* update azure-identity denpendency versions.

* clean up.

* build.yaml: specify -DskipTests before az login, as the key vault extension requires real service for testing.

* add script to delete keyvault resource.

* use reactive programing in integration test.

* remove unused import.

* apply Jianguo's comment.

* apply Zhihao's comments.

* use -Dskip.azure.test=true to skip tests that require real azure services.

* use io.quarkus.logging.Log.

* remove unused dependency.

* as we disscussed, move otherkeyvault  API test to the next PR.

* fix build failure

* sort imports used in Key Vault Extension.

* apply Ed's comments.

* add identitifier to track key vault secret clients.

* format AzureQuarkusIdentifier.

* format KeyVaultSecretClientProducer.java

* add comment for azure-identity native resources.

* fix length of application id  cannot be greater than 24
  • Loading branch information
galiacheng authored Apr 30, 2024
1 parent 122ae75 commit adf8af9
Show file tree
Hide file tree
Showing 40 changed files with 1,244 additions and 6 deletions.
22 changes: 22 additions & 0 deletions .github/build-with-maven-native.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ set -Euo pipefail
# - RESOURCE_GROUP_NAME
# - STORAGE_ACCOUNT_NAME
# - APP_CONFIG_NAME
# - KEY_VAULT_NAME

# Create a resource group
az group create \
Expand Down Expand Up @@ -71,5 +72,26 @@ credential=$(az appconfig credential list \
export QUARKUS_AZURE_APP_CONFIGURATION_ID=$(echo "${credential}" | jq -r '.id')
export QUARKUS_AZURE_APP_CONFIGURATION_SECRET=$(echo "${credential}" | jq -r '.value')

# Azure Key Vault Extension
# The same commands used in
# - integration-tests/README.md
# - integration-tests/azure-keyvault/README.md

az keyvault create \
--name ${KEY_VAULT_NAME} \
--resource-group ${RESOURCE_GROUP_NAME} \
--location eastus

az ad signed-in-user show --query id -o tsv \
| az keyvault set-policy \
--name ${KEY_VAULT_NAME} \
--object-id @- \
--secret-permissions all

export QUARKUS_AZURE_KEYVAULT_SECRET_ENDPOINT=$(az keyvault show --name ${KEY_VAULT_NAME} \
--resource-group ${RESOURCE_GROUP_NAME}\
--query properties.vaultUri\
--output tsv)

# Build native executable and run the integration tests against the Azure services
mvn -B install -Dnative -Dquarkus.native.container-build -Dnative.surefire.skip -Dazure.test=true
3 changes: 3 additions & 0 deletions .github/delete-azure-resources.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@
# - RESOURCE_GROUP_NAME
# - STORAGE_ACCOUNT_NAME
# - APP_CONFIG_NAME
# - KEY_VAULT_NAME

az appconfig delete --name ${APP_CONFIG_NAME} --resource-group ${RESOURCE_GROUP_NAME} --yes
az appconfig purge --name ${APP_CONFIG_NAME} --yes
az storage account delete --name ${STORAGE_ACCOUNT_NAME} --resource-group ${RESOURCE_GROUP_NAME} --yes
az keyvault delete --name ${KEY_VAULT_NAME} --resource-group ${RESOURCE_GROUP_NAME} --yes
az keyvault purge --name ${KEY_VAULT_NAME} --yes
az group delete --name ${RESOURCE_GROUP_NAME} --yes
14 changes: 12 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
RESOURCE_GROUP_NAME: quarkus${{ github.run_id }}${{ github.run_number }}
STORAGE_ACCOUNT_NAME: storage${{ github.run_id }}${{ github.run_number }}
APP_CONFIG_NAME: appconfig${{ github.run_id }}${{ github.run_number }}
KEY_VAULT_NAME: kv${{ github.run_id }}${{ github.run_number }}
name: Build on ${{ matrix.os }}
strategy:
fail-fast: false
Expand All @@ -67,10 +68,10 @@ jobs:
cache: 'maven'

- name: Build with Maven
run: mvn -B clean install -Dno-format
run: mvn -B clean install -Dno-format -Dskip.azure.tests=true

- name: Build with Maven (Native)
run: mvn -B install -Dnative -Dquarkus.native.container-build -Dnative.surefire.skip
run: mvn -B install -Dnative -Dquarkus.native.container-build -Dnative.surefire.skip -Dskip.azure.tests=true
if: ${{ env.AZURE_CLIENT_ID == '' || env.AZURE_TENANT_ID == '' || env.AZURE_SUBSCRIPTION_ID == '' }}

- name: Az CLI login
Expand All @@ -85,6 +86,15 @@ jobs:
run: .github/build-with-maven-native.sh
if: ${{ env.AZURE_CLIENT_ID != '' && env.AZURE_TENANT_ID != '' && env.AZURE_SUBSCRIPTION_ID != '' }}

# Fix error: Client assertion is not within its valid time range.
- name: Az CLI login
uses: azure/login@v1
with:
client-id: ${{ env.AZURE_CLIENT_ID }}
tenant-id: ${{ env.AZURE_TENANT_ID }}
subscription-id: ${{ env.AZURE_SUBSCRIPTION_ID }}
if: ${{ env.AZURE_CLIENT_ID != '' && env.AZURE_TENANT_ID != '' && env.AZURE_SUBSCRIPTION_ID != '' }}

- name: Delete Azure resources
run: .github/delete-azure-resources.sh
if: ${{ always() && env.AZURE_CLIENT_ID != '' && env.AZURE_TENANT_ID != '' && env.AZURE_SUBSCRIPTION_ID != '' }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/quarkus-snapshot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ jobs:
RESOURCE_GROUP_NAME: quarkus${{ github.run_id }}${{ github.run_number }}
STORAGE_ACCOUNT_NAME: storage${{ github.run_id }}${{ github.run_number }}
APP_CONFIG_NAME: appconfig${{ github.run_id }}${{ github.run_number }}
KEY_VAULT_NAME: kv${{ github.run_id }}${{ github.run_number }}
name: "Build against latest Quarkus snapshot"
runs-on: ubuntu-latest
# Allow <ADMIN> to manually launch the ecosystem CI in addition to the bots
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ The following extensions allows you to interact with some of the Azure Services:

- [Quarkus Azure App Configuration Extension](https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-app-configuration.html): [Azure App Configuration](https://azure.microsoft.com/products/app-configuration)
is a fast, scalable parameter storage for app configuration.
- [Quarkus Azure Key Vault Extension](https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-keyvault.html): [Azure Key Vault](https://azure.microsoft.com/products/key-vault) is a cloud service for securely storing and accessing secrets.
- [Quarkus Azure Blob Storage Extension](https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-storage-blob.html): [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs/)
is a massively scalable and secure object storage for cloud-native workloads, archives, data lakes, high-performance
computing, and machine learning.
Expand All @@ -26,6 +27,8 @@ Example applications can be found inside the [integration-tests](integration-tes

- [Azure App Configuration sample](integration-tests/azure-app-configuration): REST endpoint using the Quarkus extension
to get the configuration stored in Azure App Configuration.
- [Azure Key Vault sample](integration-tests/azure-keyvault): REST endpoint using the Quarkus extension
to create secret via [SecretClient](https://learn.microsoft.com/java/api/com.azure.security.keyvault.secrets.secretclient) and [SecretAsyncClient](https://learn.microsoft.com/java/api/com.azure.security.keyvault.secrets.secretasyncclient) in Azure Key Vault.
- [Azure Blob Storage sample](integration-tests/azure-storage-blob): REST endpoint using the Quarkus extension to
upload and download files to/from Azure Blob Storage.

Expand Down
31 changes: 29 additions & 2 deletions bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@
<azure-sdk-bom.version>1.2.22</azure-sdk-bom.version>
<azure.core.http.client.vertx.version>1.0.0-beta.17</azure.core.http.client.vertx.version>
<!-- @sync com.azure:azure-identity:${azure-identity.version} dep:com.microsoft.azure:msal4j -->
<msal4j.version>1.15.0</msal4j.version>
<msal4j.version>1.14.3</msal4j.version>
<!-- @sync com.azure:azure-identity:${azure-identity.version} dep:net.minidev:json-smart -->
<json-smart.verion>2.5.0</json-smart.verion>
<!-- needed for dependency convergence until Azure SDK is upgraded -->
<woodstox-core.version>6.6.2</woodstox-core.version>
</properties>
Expand All @@ -40,7 +42,12 @@
<type>pom</type>
<scope>import</scope>
</dependency>


<dependency>
<groupId>net.minidev</groupId>
<artifactId>json-smart</artifactId>
<version>${json-smart.verion}</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>msal4j</artifactId>
Expand All @@ -67,6 +74,16 @@
</dependency>

<!-- Azure Services Extensions -->
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-identity</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-identity-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-core</artifactId>
Expand Down Expand Up @@ -107,6 +124,16 @@
<artifactId>quarkus-azure-jackson-dataformat-xml-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-keyvault</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-keyvault-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-storage-blob</artifactId>
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
* xref:quarkus-azure-app-configuration.adoc[Quarkus Azure App Configuration Extension]
* xref:quarkus-azure-keyvault.adoc[Quarkus Azure Key Vault Extension]
* xref:quarkus-azure-storage-blob.adoc[Quarkus Azure Blob Storage Extension]
29 changes: 29 additions & 0 deletions docs/modules/ROOT/pages/includes/quarkus-azure-keyvault.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

:summaryTableId: quarkus-azure-keyvault
[.configuration-legend]
icon:lock[title=Fixed at build time] Configuration property fixed at build time - All other configuration properties are overridable at runtime
[.configuration-reference.searchable, cols="80,.^10,.^10"]
|===

h|[[quarkus-azure-keyvault_configuration]]link:#quarkus-azure-keyvault_configuration[Configuration property]

h|Type
h|Default

a| [[quarkus-azure-keyvault_quarkus.quarkus.azure.keyvault.secret.endpoint]]`link:#quarkus-azure-keyvualt_quarkus.azure.keyvault.secret.endpoint[quarkus.azure.keyvault.secret.endpoint]`


[.description]
--
The endpoint of Azure Key Vault.

ifdef::add-copy-button-to-env-var[]
Environment variable: env_var_with_copy_button:+++QUARKUS_AZURE_KEYVAULT_SECRET_ENDPOINT+++[]
endif::add-copy-button-to-env-var[]
ifndef::add-copy-button-to-env-var[]
Environment variable: `+++QUARKUS_AZURE_KEYVAULT_SECRET_ENDPOINT+++`
endif::add-copy-button-to-env-var[]
--|string
|required icon:exclamation-circle[title=Configuration property is required]

|===
2 changes: 2 additions & 0 deletions docs/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,6 @@ include::./includes/attributes.adoc[]

include::quarkus-azure-app-configuration.adoc[leveloffset=+1, opts=optional]

include::quarkus-azure-keyvault.adoc[leveloffset=+1, opts=optional]

include::quarkus-azure-storage-blob.adoc[leveloffset=+1, opts=optional]
167 changes: 167 additions & 0 deletions docs/modules/ROOT/pages/quarkus-azure-keyvault.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
= Quarkus Azure Key Vault Extension

include::./includes/attributes.adoc[]

include::./includes/support.adoc[]

https://azure.microsoft.com/en-us/products/key-vault[Azure Key Vault] is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys.
This extension allows you to create and retrieve secret from Azure Key Vault by injecting the following object inside your Quarkus application.

* `com.azure.security.keyvault.secrets.SecretClient`
* `com.azure.security.keyvault.secrets.SecretAsyncClient`
The extension produces SecretClient and SecretAsyncClient using https://learn.microsoft.com/java/api/overview/azure/identity-readme#defaultazurecredential[DefaultAzureCredential].
Developers who want more control or whose scenario isn't served by the default settings should build client using other credential types.

== Installation

If you want to use this extension, you need to add the `io.quarkiverse.azureservices:quarkus-azure-services` extension first to your build file.

For instance, with Maven, add the following dependency to your POM file:

[source,xml,subs=attributes+]
----
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-keyvault</artifactId>
<version>{project-version}</version>
</dependency>
----

== How to Use It

Once you have added the extension to your project, follow the next steps, so you can inject `com.azure.security.keyvault.secrets.SecretClient` or `com.azure.security.keyvault.secrets.SecretAsyncClient` object in your application to manage secret.

=== Setup your Azure Environment

include::includes/azure-setup-environment.adoc[]

Create an Azure resource group with the `az group create` command.
A resource group is a logical container into which Azure resources are deployed and managed.

[source,shell]
----
az group create \
--name rg-quarkus-azure-keyvault \
--location eastus
----

Create a general-purpose key vault with the following command:

[source,shell]
----
az keyvault create --name kvquarkusazurekv0423 \
--resource-group rg-quarkus-azure-keyvault \
--location eastus
----

Key Vault provides secure storage of generic secrets, such as passwords and database connection strings.
All secrets in your key vault are stored encrypted. The Azure Key Vault service encrypts your secrets when you add them, and decrypts them automatically when you read them.

Access Control for secrets managed in Key Vault, is provided at the level of the Key Vault.
The following command uses your Azure AD account to authorize the operation to manage secret.
Even if you are the key vault owner, you need explicit permissions to perform operations against secret.

Assign all secret permissions(backup, delete, get, list, purge, recover, restore, set) to yourself:

[source,shell]
----
az ad signed-in-user show --query id -o tsv \
| az keyvault set-policy \
--name kvquarkusazurekv0423 \
--object-id @- \
--secret-permissions all
----

If you log into the http://portal.azure.com/[Azure portal], you can see the key vault you created. Select **Objects** -> **Secrets**, you will find the Secrets page.

image::quarkus-azure-keyvault-secret-portal.png[alt=Azure Portal showing Key Vault Secrets]

=== Configure the Azure Key Vault Secret Client

As you can see below in the _Configuration Reference_ section, the configuration option `quarkus.azure.keyvault.secret.endpoint` is mandatory.
To get the endpoint, execute the following Azure CLI command:

[source,shell]
----
az keyvault show --name kvquarkusazurekv0423 \
--resource-group rg-quarkus-azure-keyvault \
--query properties.vaultUri \
--output tsv
----

Then, in the `application.properties` file, add the following property:

[source,properties]
----
quarkus.azure.keyvault.secret.endpoint=https://kvquarkusazurekv0423.vault.azure.net/
----

=== Inject the Azure Key Vault Secret Client

Now that your Azure environment is ready and that you have configured the extension, you can inject the `com.azure.security.keyvault.secrets.SecretClient` object in your application, so you can interact with Azure Key Vault Secret.

This is a https://quarkus.io/guides/command-mode-reference[Quarkus CLI application]. The application will:

* Ask for a secret value.
* Create a secret with name `mySecret` and set its value.
* Retrieve and print the secret value.
* Delete the secret.

You can build and run the application in development mode using command:

```
quarkus dev
```

[source,java]
----
@QuarkusMain
public class KeyVaultSecretApplication implements QuarkusApplication {
@Inject
SecretClient secretClient;
@Override
public int run(String... args) throws Exception {
Console con = System.console();
String secretName = "mySecret";
System.out.println("Create secret: " + secretName);
System.out.println("Please provide the value of your secret > ");
String secretValue = con.readLine();
System.out.println("Creating a secret called '" + secretName + "' with value '" + secretValue + "' ... ");
secretClient.setSecret(new KeyVaultSecret(secretName, secretValue));
System.out.println("Retrieving your secret...");
KeyVaultSecret retrievedSecret = secretClient.getSecret(secretName);
System.out.println("Your secret's value is '" + retrievedSecret.getValue() + "'.");
System.out.println("Deleting your secret ... ");
SyncPoller<DeletedSecret, Void> deletionPoller = secretClient.beginDeleteSecret(secretName);
deletionPoller.waitForCompletion();
System.out.println("done.");
return 0;
}
}
----

After running the application, if you log into the http://portal.azure.com/[Azure portal], you can see the key vault and the secret you created.
As the secret is deleted, you will find the secret from **Objects** -> **Secrets** -> **Manage deleted secrets**.

image::quarkus-azure-keyvault-deleted-secret-portal.png[alt=Azure Portal showing the deleted secrets]

You can also inject `com.azure.security.keyvault.secrets.SecretAsyncClient` object to your application. For more usage, see https://learn.microsoft.com/java/api/com.azure.security.keyvault.secrets.secretasyncclient?view=azure-java-stable[com.azure.security.keyvault.secrets.secretasyncclient].

== Extension Configuration Reference

include::includes/quarkus-azure-keyvault.adoc[leveloffset=+1, opts=optional]
Loading

0 comments on commit adf8af9

Please sign in to comment.