HMS Federation Documentation (#3688)

Added documentation for HMS Fed

docs/ucx/docs/process/index.mdx

@@ -8,7 +8,8 @@ On a high level, the steps in migration process are:
 1. [assessment](/docs/reference/workflows#assessment-workflow)
 2. [group migration](/docs/reference/workflows#group-migration-workflow)
 3. [table migration](/docs/process/#table-migration-process)
-4. [data reconciliation](/docs/reference/workflows#post-migration-data-reconciliation-workflow)
+4. [HMS Federation](/docs/process/#hms-federation)
+5. [data reconciliation](/docs/reference/workflows#post-migration-data-reconciliation-workflow)
 6. [code migration](/docs/reference/commands#code-migration-commands)
 7. [delta live table pipeline migration](/docs/process#delta-live-table-pipeline-migration-process)
 8. [final details](#final-details)
@@ -290,6 +291,26 @@ The [`revert-migrated-tables` command](/docs/reference/commands#revert-migrated-
 the `upgraded_to` property on the source object. Use this command to allow for migrating a table or view again.
 
 
+## HMS Federation
+
+Hive Metastore (HMS) Federation is a feature that allows HMS to be federated to a catalog.
+More information is available in the [Databricks documentation](https://docs.databricks.com/en/data-governance/unity-catalog/hms-federation/index.html).
+HMS Federation is used in two scenarios:
+1. A step to ease the migration to Unity Catalog.
+2. A Hybrid solution where both HMS and UC access to the data is required.
+
+> You are required to complete the [assessment workflow](/docs/reference/workflows#assessment-workflow) before starting the table migration workflow.
+HMS Federation is an alternative to table migration.
+You are required to complete all the steps documented in the [table migration process](/docs/process#table-migration-process) except for:
+- [Table Mapping](/docs/process#table-mapping)
+- [Create Catalogs and Schemas](/docs/process/#step-2-create-catalogs-and-schemas)
+- [Migrate Hive metastore data objects](/docs/process#migrate-hive-metastore-data-objects)
+
+There are two steps to enable HMS Federation:
+1. [enable-hms-federation](/docs/reference/commands#enable-hms-federation)
+2. [create-federated-catalog](/docs/reference/commands#create-federated-catalog)
+
+
 ## Code Migration
 
 After you're done with the [table migration](#table-migration-process) and

docs/ucx/docs/reference/commands/index.mdx

@@ -667,10 +667,53 @@ These commands are for [pipeline migration](/docs/process#delta-live-table-pipel
 
 ### `migrate-dlt-pipelines`
 
-```text
+```commandline
 $ databricks labs ucx migrate-dlt-pipelines [--include-pipeline-ids <comma separated list of pipeline ids>] [--exclude-pipeline-ids <comma separated list of pipeline ids>]
 ```
 
+## HMS (Hive Metastore) federation commands
+
+These commands are for [Hive Metastore federation](/docs/process#hive-metastore-federation-process) and require the [assessment workflow](/docs/reference/workflows#assessment-workflow) to be completed.
+Hive Metastore federation creates a UC catalog that mirrors all the schemas and tables in the source Hive Metastore. It is an alternative to the [table migration process](/docs/process#table-migration-process).
+
+### `enable_hms_federation`
+
+```commandline
+$ databricks labs ucx enable_hms_federation
+```
+
+This command sets the `enable_hms_federation` flag in the config.yml file to `True`.<br/>
+It is used to enable the Hive Metastore federation process.
+It is also used by the `migrate-locations` command to determine whether to set the [Fallback Mode](https://docs.databricks.com/en/data-governance/unity-catalog/hms-federation/hms-federation-internal.html) to "True" in the newly created external locations.
+In AWS it also also allows the creation a GLUE role if one is required using the `create-missing-principals` command.
+
+### `create_federated_catalog`
+
+```commandline
+$ databricks labs ucx create_federated_catalog
+```
+
+This command performs the following steps:
+1. Prompts the user for a name for the federated catalog and connection.
+2. Create an HMS "Connection" in the Unity Catalog. The HMS connection will point to the HMS used by the workspace that was identified in the installation process. It supports:
+        - Internal Workspace Hive Metastore
+        - AWS Glue Data Catalog
+        - External Database Hive Metastore (Postgres, SQL Server, MySQL)
+3. Create a federated catalog in Unity Catalog. The federated catalog will mirror all the schemas and tables in the source Hive Metastore.
+4. Whitelist the external location identified in the "Assessment" phase.
+
+### `migrate-glue-credentials`
+
+```commandline
+$ databricks labs ucx migrate-glue-credentials
+```
+
+AWS Only.
+This command is used for HMS Federation and creates a UC Service Credential for GLUE. It requires an existing UC compatible GLUE role.
+The role can be created using the `create-missing-principals` command.
+
+
+
 ## Utility commands
 
 ### `logs`