doc: update find_secret on Fabric and doc (#2132)

* add find secret doc for synapse and adb * Apply suggestions from code review * update find_secret function * minor fix * fix style * update find_secret for fabric * style * fix find_secret e2e test * Update Platform.py * fix find_secret ---------
microsoft · Nov 27, 2023 · 2a01c8e · 2a01c8e
1 parent 23222c0
commit 2a01c8e
Show file tree

Hide file tree

Showing 2 changed files with 160 additions and 1 deletion.
diff --git a/core/src/main/python/synapse/ml/core/platform/Platform.py b/core/src/main/python/synapse/ml/core/platform/Platform.py
@@ -47,10 +47,15 @@ def running_on_databricks():
 
 
 def find_secret(secret_name, keyvault):
-    if running_on_synapse() or running_on_synapse_internal():
+    if running_on_synapse():
         from notebookutils.mssparkutils.credentials import getSecret
 
         return getSecret(keyvault, secret_name)
+    elif running_on_synapse_internal():
+        from notebookutils.mssparkutils.credentials import getSecret
+
+        keyVaultURL = f"https://{keyvault}.vault.azure.net/"
+        return getSecret(keyVaultURL, secret_name)
     elif running_on_databricks():
         from pyspark.sql import SparkSession
         from pyspark.dbutils import DBUtils

diff --git a/docs/Get Started/Set up Cognitive Services.ipynb b/docs/Get Started/Set up Cognitive Services.ipynb
@@ -64,6 +64,160 @@
     "* [Create Azure AI service (Decision, Language, Speech, Vision) resource](https://ms.portal.azure.com/#create/Microsoft.CognitiveServicesAllInOne). You can follow the steps at [Create a multi-service resource for Azure AI services](https://learn.microsoft.com/en-us/azure/ai-services/multi-service-resource?tabs=windows&pivots=azportal#create-a-new-azure-cognitive-services-resource). \n",
     "* Get your Azure AI Services resource's key. After your resource is successfully deployed, select **Next Steps** > **Go to resource**. Once at the resource, you can get the key from **Resource Management** > **Keys and Endpoint**. Copy the key and paste it into the notebook. Store keys securely and do not share them. "
    ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Manage secrets with Azure Key Vault and access the secrets with find_secret"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "After you create an Azure AI resource, you will obtain a resource key. You can use this resource key directly in our notebooks as a string, but we recommend to manage secrets with Azure Key Vault. Azure Key Vault is a cloud-based service that allows you to store and manage cryptographic keys, certificates, and secrets used by cloud applications and services.\n",
+    "\n",
+    "You can skip the following content if you want to use a secret string instead of Azure Key Vault. This is not recommended for production workloads."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Create Azure Key Vault\n",
+    "\n",
+    "Refer to [this article](https://learn.microsoft.com/azure/key-vault/general/quick-create-portal) to create a key vault using Azure Portal"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Save secret to Azure Key Vault\n",
+    "\n",
+    "- Go to Access control (IAM) and assign **Key Vault Administrator Role** to yourself.\n",
+    "- Go to Secrets and choose **+ Generate/Import**, create a key with the secret value obtained from Azure AI service.\n",
+    "- Choose **Create**."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Use find_secret on Microsoft Fabric / Power BI\n",
+    "\n",
+    "- Go to Azure Key Vault, Select Access control (IAM), Select **+ Add**, Add role assignment.\n",
+    "- Granted the Fabric or Power BI Service Account Get permissions in the Azure Key Vault."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Use find_secret on Azure Databricks\n",
+    "\n",
+    "- In the Azure Portal, find and select the Azure Key Vault Instance. Select the **Access configuration** tab under Settings. Set Permission model to Vault access policy.\n",
+    "- On Databricks, go to `https://<databricks-instance>#secrets/createScope`. This URL is case sensitive.\n",
+    " \n",
+    "\n",
+    "<img src=\"https://mmlspark.blob.core.windows.net/graphics/adb_create_secret_scope.png\" width=\"600\" style=\"float: center;\"/>\n",
+    "\n",
+    "\n",
+    "\n",
+    "- Enter the name of the secret scope and choose desired Manage Principal.\n",
+    "- Go to your Azure Key Vault -> Properties and find the **DNS Name** (Vault URI on Azure Key Vault) and **Resource ID**. Enter the DNS Name and Resource ID on Databricks createScope page.\n",
+    "\n",
+    "\n",
+    "<img src=\"https://mmlspark.blob.core.windows.net/graphics/adb_find_resource_id.png\" width=\"600\" style=\"float: center;\"/>\n",
+    "\n",
+    "\n",
+    "- Click the **Create** Button.\n",
+    " \n",
+    "\n",
+    "Refer to [this article](https://learn.microsoft.com/azure/databricks/security/secrets/secret-scopes#--create-an-azure-key-vault-backed-secret-scope) for more details.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Use find_secret on Synapse\n",
+    "\n",
+    "- Open the Synapse Studio and go to the Manage tab.\n",
+    "- Under External connections, select Linked services.\n",
+    "- To add a linked service, select New.\n",
+    "- Select the Azure Key Vault tile from the list and select Continue.\n",
+    "- Enter a linked service name and choose the key vault you want to connect to.\n",
+    "- Select Create\n",
+    "- Click Publish\n",
+    "\n",
+    "\n",
+    "- Go to Azure Key Vault, Select Access control (IAM), Select **+ Add**, Add role assignment.\n",
+    "- Choose **Key Vault Administrator** in Role blade, select Next.\n",
+    "- In Members blade, choose Assign access to **Managed identity**. Select members, choose the subscription your Synapse Workspace in. For Managed identity, select Synapse workspace, choose your workspace.\n",
+    "- Select **Review + assign**."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Quick Test"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "vscode": {
+     "languageId": "plaintext"
+    }
+   },
+   "outputs": [],
+   "source": [
+    "from synapse.ml.services.language import AnalyzeText\n",
+    "from synapse.ml.core.platform import find_secret\n",
+    "\n",
+    "ai_service_key = find_secret(\n",
+    "    secret_name=\"ai-services-api-key\", keyvault=\"mmlspark-build-keys\"\n",
+    ")  # use your own key vault name and api name\n",
+    "ai_service_location = \"eastus\"  # use your own AI service location\n",
+    "\n",
+    "df = spark.createDataFrame(\n",
+    "    data=[\n",
+    "        [\"en\", \"Dr. Smith has a very modern medical office, and she has great staff.\"],\n",
+    "        [\"en\", \"I had a wonderful trip to Seattle last week.\"],\n",
+    "    ],\n",
+    "    schema=[\"language\", \"text\"],\n",
+    ")\n",
+    "\n",
+    "entity_recognition = (\n",
+    "    AnalyzeText()\n",
+    "    .setKind(\"EntityRecognition\")\n",
+    "    .setLocation(ai_service_location)\n",
+    "    .setSubscriptionKey(ai_service_key)\n",
+    "    .setTextCol(\"text\")\n",
+    "    .setOutputCol(\"entities\")\n",
+    "    .setErrorCol(\"error\")\n",
+    "    .setLanguageCol(\"language\")\n",
+    ")\n",
+    "\n",
+    "df_results = entity_recognition.transform(df)\n",
+    "display(df_results.select(\"language\", \"text\", \"entities.documents.entities\"))"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Your result should looks like\n",
+    "\n",
+    "| language         | text     | entities |\n",
+    "|--------------|-----------|------------|\n",
+    "| en           | Dr. Smith has a very modern medical office, and she has great staff.      | [{\"category\": \"Person\", \"confidenceScore\": 0.98, \"length\": 5, \"offset\": 4, \"subcategory\": null, \"text\": \"Smith\"}, {\"category\": \"Location\", \"confidenceScore\": 0.79, \"length\": 14, \"offset\": 28, \"subcategory\": \"Structural\", \"text\": \"medical office\"}, {\"category\": \"PersonType\", \"confidenceScore\": 0.85, \"length\": 5, \"offset\": 62, \"subcategory\": null, \"text\": \"staff\"}]        |\n",
+    "| en           | I had a wonderful trip to Seattle last week.  | [{\"category\": \"Event\", \"confidenceScore\": 0.74, \"length\": 4, \"offset\": 18, \"subcategory\": null, \"text\": \"trip\"}, {\"category\": \"Location\", \"confidenceScore\": 1, \"length\": 7, \"offset\": 26, \"subcategory\": \"GPE\", \"text\": \"Seattle\"}, {\"category\": \"DateTime\", \"confidenceScore\": 0.8, \"length\": 9, \"offset\": 34, \"subcategory\": \"DateRange\", \"text\": \"last week\"}]       |"
+   ]
   }
  ],
  "metadata": {