Skip to content

Commit

Permalink
Added Process Documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
FastLee committed Feb 13, 2025
1 parent b56ccc9 commit 568e2a4
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 3 deletions.
23 changes: 22 additions & 1 deletion docs/ucx/docs/process/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,8 @@ On a high level, the steps in migration process are:
1. [assessment](/docs/reference/workflows#assessment-workflow)
2. [group migration](/docs/reference/workflows#group-migration-workflow)
3. [table migration](/docs/process/#table-migration-process)
4. [data reconciliation](/docs/reference/workflows#post-migration-data-reconciliation-workflow)
4. [HMS Federation](/docs/process/#hms-federation)
5. [data reconciliation](/docs/reference/workflows#post-migration-data-reconciliation-workflow)
6. [code migration](/docs/reference/commands#code-migration-commands)
7. [delta live table pipeline migration](/docs/process#delta-live-table-pipeline-migration-process)
8. [final details](#final-details)
Expand Down Expand Up @@ -290,6 +291,26 @@ The [`revert-migrated-tables` command](/docs/reference/commands#revert-migrated-
the `upgraded_to` property on the source object. Use this command to allow for migrating a table or view again.


## HMS Federation

Hive Metastore (HMS) Federation is a feature that allows HMS to be federated to a catalog.
More information is available in the [Databricks documentation](https://docs.databricks.com/en/data-governance/unity-catalog/hms-federation/index.html).
HMS Federation is used in two scenarios:
1. A step to ease the migration to Unity Catalog.
2. A Hybrid solution where both HMS and UC access to the data is required.

> You are required to complete the [assessment workflow](/docs/reference/workflows#assessment-workflow) before starting the table migration workflow.
HMS Federation is an alternative to table migration.
You are required to complete all the steps documented in the [table migration process](/docs/process#table-migration-process) except for:
- [Table Mapping](/docs/process#table-mapping)
- [Create Catalogs and Schemas](/docs/process/#step-2-create-catalogs-and-schemas)
- [Migrate Hive metastore data objects](/docs/process#migrate-hive-metastore-data-objects)

There are two steps to enable HMS Federation:
1. [enable-hms-federation](/docs/reference/commands#enable-hms-federation)
2. [create-federated-catalog](/docs/reference/commands#create-federated-catalog)


## Code Migration

After you're done with the [table migration](#table-migration-process) and
Expand Down
5 changes: 3 additions & 2 deletions docs/ucx/docs/reference/commands/index.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -682,8 +682,9 @@ Hive Metastore federation creates a UC catalog that mirrors all the schemas and
$ databricks labs ucx enable_hms_federation
```

This command sets the `enable_hms_federation` flag in the config.yml file to `True`. This flag is used to enable the Hive Metastore federation process.
This flag is used by the `migrate-locations` command to determine whether to set the [Fallback Mode](https://docs.databricks.com/en/data-governance/unity-catalog/hms-federation/hms-federation-internal.html) to "True" in the newly created external locations.
This command sets the `enable_hms_federation` flag in the config.yml file to `True`.<br/>
It is used to enable the Hive Metastore federation process.
It is also used by the `migrate-locations` command to determine whether to set the [Fallback Mode](https://docs.databricks.com/en/data-governance/unity-catalog/hms-federation/hms-federation-internal.html) to "True" in the newly created external locations.
In AWS it also also allows the creation a GLUE role if one is required using the `create-missing-principals` command.

### `create_federated_catalog`
Expand Down

0 comments on commit 568e2a4

Please sign in to comment.