Set catalog on SchemaDeployer to overwrite the default hive_metastore
#296
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
✅ 35/35 passed, 4 skipped, 9m58s total Running from acceptance #419 |
JCZuurmond
had a problem deploying
to
runtime
GitHub Actions
Failure
JCZuurmond
had a problem deploying
to
runtime
GitHub Actions
Failure
JCZuurmond
commented
Sep 26, 2024
| self, | ||
| sql_backend: SqlBackend, | ||
| inventory_schema: str, | ||
| mod: Any, |
There was a problem hiding this comment.
Also, I would remove this or make it optional as views are not always required and could be supplied in different ways (like path to SQL file or SQL as text), but also not changing in this PR
nfx
requested changes
Sep 26, 2024
JCZuurmond
had a problem deploying
to
runtime
GitHub Actions
Failure
JCZuurmond
commented
Sep 26, 2024
| @@ -140,7 +140,7 @@ class StatementExecutionExt: | |||
| megabytes or gigabytes of data serialized in Apache Arrow format, and low result fetching latency, should use | |||
| the stateful Databricks SQL Connector for Python.""" | |||
|
|
|||
| def __init__( # pylint: disable=too-many-arguments | |||
| def __init__( # pylint: disable=too-many-arguments,too-many-positional-arguments | |||
There was a problem hiding this comment.
No lint CI fails on this line, which was required by the linter
7 tasks
nfx
added a commit
that referenced
this pull request
Sep 26, 2024
* Bump actions/checkout from 4.1.7 to 4.2.0 ([#295](#295)). In this version 4.2.0 release of the `actions/checkout` library, the team has added `Ref` and `Commit` outputs, which provide the ref and commit that were checked out, respectively. The update also includes dependency updates to `braces`, `minor-npm-dependencies`, `docker/build-push-action`, and `docker/login-action`, all of which were automatically resolved by Dependabot. These updates improve compatibility and stability for users of the library. This release is a result of contributions from new team members @yasonk and @lucacome. Users can find a detailed commit history, pull requests, and release notes in the associated links. The team strongly encourages all users to upgrade to this new version to access the latest features and improvements. * Set catalog on `SchemaDeployer` to overwrite the default `hive_metastore` ([#296](#296)). In this release, the default catalog for `SchemaDeployer` has been changed from `hive_metastore` to a user-defined catalog, allowing for more flexibility in deploying resources to different catalogs. A new dependency, `databricks-labs-pytester`, has been added with a version constraint of `>=0.2.1`, which may indicate the introduction of new testing functionality. The `SchemaDeployer` class has been updated to accept a `catalog` parameter and the tests for deploying and deleting schemas, tables, and views have been updated to reflect these changes. The `test_deploys_schema`, `test_deploys_dataclass`, and `test_deploys_view` tests have been updated to accept a `inventory_catalog` parameter, and the `caplog` fixture is used to capture log messages and assert that they contain the expected messages. Additionally, a new test function `test_statement_execution_backend_overwrites_table` has been added to the `tests/integration/test_backends.py` file to test the functionality of the `StatementExecutionBackend` class in overwriting a table in the database and retrieving the correct data. Issue [#294](#294) has been resolved, and progress has been made on issue [#278](#278), but issue [#280](#280) has been marked as technical debt and issue [#287](#287) is required for the CI to pass. Dependency updates: * Bump actions/checkout from 4.1.7 to 4.2.0 ([#295](#295)).
Merged
nfx
added a commit
that referenced
this pull request
Sep 26, 2024
* Bump actions/checkout from 4.1.7 to 4.2.0 ([#295](#295)). In this version 4.2.0 release of the `actions/checkout` library, the team has added `Ref` and `Commit` outputs, which provide the ref and commit that were checked out, respectively. The update also includes dependency updates to `braces`, `minor-npm-dependencies`, `docker/build-push-action`, and `docker/login-action`, all of which were automatically resolved by Dependabot. These updates improve compatibility and stability for users of the library. This release is a result of contributions from new team members @yasonk and @lucacome. Users can find a detailed commit history, pull requests, and release notes in the associated links. The team strongly encourages all users to upgrade to this new version to access the latest features and improvements. * Set catalog on `SchemaDeployer` to overwrite the default `hive_metastore` ([#296](#296)). In this release, the default catalog for `SchemaDeployer` has been changed from `hive_metastore` to a user-defined catalog, allowing for more flexibility in deploying resources to different catalogs. A new dependency, `databricks-labs-pytester`, has been added with a version constraint of `>=0.2.1`, which may indicate the introduction of new testing functionality. The `SchemaDeployer` class has been updated to accept a `catalog` parameter and the tests for deploying and deleting schemas, tables, and views have been updated to reflect these changes. The `test_deploys_schema`, `test_deploys_dataclass`, and `test_deploys_view` tests have been updated to accept a `inventory_catalog` parameter, and the `caplog` fixture is used to capture log messages and assert that they contain the expected messages. Additionally, a new test function `test_statement_execution_backend_overwrites_table` has been added to the `tests/integration/test_backends.py` file to test the functionality of the `StatementExecutionBackend` class in overwriting a table in the database and retrieving the correct data. Issue [#294](#294) has been resolved, and progress has been made on issue [#278](#278), but issue [#280](#280) has been marked as technical debt and issue [#287](#287) is required for the CI to pass. Dependency updates: * Bump actions/checkout from 4.1.7 to 4.2.0 ([#295](#295)).
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Set catalog on
SchemaDeployerto overwrite the defaulthive_metastoreLinked issues
Resolves #294
Needs #280 (tech debt to tackle later)
Progresses #278
Requires #287 for the CI to pass