Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v0.2.1 #46

Merged
merged 1 commit into from
Sep 18, 2024
Merged

Release v0.2.1 #46

merged 1 commit into from
Sep 18, 2024

Conversation

nfx
Copy link
Collaborator

@nfx nfx commented Sep 18, 2024

  • Moved remaining UCX integration tests and fixtures (#45). In this release, we have made significant changes to the UCX integration tests and fixtures, as indicated by multiple commit messages. Firstly, we have moved remaining UCX integration tests and fixtures, introducing a new PyTest fixture called Installation in the README.md file, providing instructions on how to add databricks-labs-pytester as a test-time dependency when using hatch as the build system. Additionally, we have added the make_feature_table fixture, which creates a Databricks feature table and cleans it up after the test, taking optional parameters for customization. We have also modified the mypy configuration in the pyproject.toml file to allow untyped imports during the type-checking process. In the compute.py file, we have updated the make_job fixture to return a function that creates a databricks.sdk.service.jobs.Job instance, and modified the create function to return the databricks.sdk.service.jobs.Job instance directly. We have also added a new fixture called make_feature_table in the plugin file, which simulates the lifecycle of a feature table in the machine learning service, with functions to generate a unique name and create/remove the feature table. In the test_catalog.py file, we have made changes to clean up the file and ensure proper logging of test events and errors. Overall, these changes aim to refactor, expand functionality, and improve user-friendliness for the adopters of the project, ensuring proper logging and debugging capabilities.
  • [internal] port over existing UCX integration tests (#44). Three new integration tests have been added to the UCX project to verify the functionality of the RemoveAfter property for tables and schemas. The test_remove_after_property_table and test_remove_after_property_schema tests create new tables and schemas, respectively, and check if the RemoveAfter property is included in their properties. However, these tests are still marked as TODO due to existing issues with the tables.get and schemas.get functions. In addition, existing UCX integration tests have been ported over, which include new functions for testing the removal of resources based on the RemoveAfter tag. These tests are located in the tests/integration/fixtures/test_compute.py file and test the removal of various types of resources, including jobs, clusters, warehouses, and instance pools. The tests ensure that the time until purge is less than the TEST_RESOURCE_PURGE_TIMEOUT value plus one hour and import the datetime module and the TEST_RESOURCE_PURGE_TIMEOUT constant from the watchdog fixture, as well as the logging and databricks.sdk.service.iam modules.

* Moved remaining UCX integration tests and fixtures ([#45](#45)). In this release, we have made significant changes to the UCX integration tests and fixtures, as indicated by multiple commit messages. Firstly, we have moved remaining UCX integration tests and fixtures, introducing a new PyTest fixture called `Installation` in the README.md file, providing instructions on how to add `databricks-labs-pytester` as a test-time dependency when using `hatch` as the build system. Additionally, we have added the `make_feature_table` fixture, which creates a Databricks feature table and cleans it up after the test, taking optional parameters for customization. We have also modified the `mypy` configuration in the `pyproject.toml` file to allow untyped imports during the type-checking process. In the `compute.py` file, we have updated the `make_job` fixture to return a function that creates a `databricks.sdk.service.jobs.Job` instance, and modified the `create` function to return the `databricks.sdk.service.jobs.Job` instance directly. We have also added a new fixture called `make_feature_table` in the plugin file, which simulates the lifecycle of a feature table in the machine learning service, with functions to generate a unique name and create/remove the feature table. In the `test_catalog.py` file, we have made changes to clean up the file and ensure proper logging of test events and errors. Overall, these changes aim to refactor, expand functionality, and improve user-friendliness for the adopters of the project, ensuring proper logging and debugging capabilities.
* [internal] port over existing UCX integration tests ([#44](#44)). Three new integration tests have been added to the UCX project to verify the functionality of the `RemoveAfter` property for tables and schemas. The `test_remove_after_property_table` and `test_remove_after_property_schema` tests create new tables and schemas, respectively, and check if the `RemoveAfter` property is included in their properties. However, these tests are still marked as `TODO` due to existing issues with the `tables.get` and `schemas.get` functions. In addition, existing UCX integration tests have been ported over, which include new functions for testing the removal of resources based on the `RemoveAfter` tag. These tests are located in the `tests/integration/fixtures/test_compute.py` file and test the removal of various types of resources, including jobs, clusters, warehouses, and instance pools. The tests ensure that the time until purge is less than the `TEST_RESOURCE_PURGE_TIMEOUT` value plus one hour and import the `datetime` module and the `TEST_RESOURCE_PURGE_TIMEOUT` constant from the `watchdog` fixture, as well as the `logging` and `databricks.sdk.service.iam` modules.
@nfx nfx merged commit 0fa2127 into main Sep 18, 2024
7 of 9 checks passed
@nfx nfx deleted the prepare/0.2.1 branch September 18, 2024 12:40
Copy link

This PR breaks backwards compatibility for databrickslabs/blueprint downstream. See build logs for more details.

Running from downstreams #20

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant