Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added make_run_as fixture #82

Merged
merged 7 commits into from
Nov 15, 2024
Merged

Added make_run_as fixture #82

merged 7 commits into from
Nov 15, 2024

Conversation

nfx
Copy link
Collaborator

@nfx nfx commented Nov 14, 2024

make_run_as fixture

This fixture provides a function to create an account service principal via acc fixture and
assign it to a workspace. The service principal is removed after the test is complete. The service principal is
created with a random display name and assigned to the workspace with the default permissions.

Use the account_groups argument to assign the service principal to account groups, which have the required
permissions to perform a specific action.

Example:

def test_run_as_lower_privilege_user(make_run_as, ws):
    run_as = make_run_as(account_groups=['account.group.name'])
    through_query = next(run_as.sql_fetch_all("SELECT CURRENT_USER() AS my_name"))
    me = ws.current_user.me()
    assert me.user_name != through_query.my_name

Returned object has the following properties:

  • ws: Workspace client that is authenticated as the ephemeral service principal.
  • sql_backend: SQL backend that is authenticated as the ephemeral service principal.
  • sql_exec: Function to execute a SQL statement on behalf of the ephemeral service principal.
  • sql_fetch_all: Function to fetch all rows from a SQL statement on behalf of the ephemeral service principal.
  • display_name: Display name of the ephemeral service principal.
  • application_id: Application ID of the ephemeral service principal.
  • if you want to have other fixtures available in the context of the ephemeral service principal, you can override
    the ws fixture on the file level, which would make all workspace fixtures provided by this
    plugin to run as lower privilege ephemeral service principal. You cannot combine it with the account-admin-level
    principal you're using to create the ephemeral principal.

Example:

from pytest import fixture

@fixture
def ws(make_run_as):
    run_as = make_run_as(account_groups=['account.group.used.for.all.tests.in.this.file'])
    return run_as.ws

def test_creating_notebook_on_behalf_of_ephemeral_principal(make_notebook):
    notebook = make_notebook()
    assert notebook.exists()

See also acc, ws, make_random, env_or_skip, log_account_link.

@nfx nfx requested a review from JCZuurmond November 14, 2024 19:18
@nfx nfx marked this pull request as draft November 14, 2024 19:18
Copy link

github-actions bot commented Nov 14, 2024

✅ 38/38 passed, 6 skipped, 4m59s total

Running from acceptance #135

@nfx nfx marked this pull request as ready for review November 15, 2024 12:07
@nfx nfx temporarily deployed to account-admin November 15, 2024 13:21 — with GitHub Actions Inactive
@nfx nfx merged commit 60f4f6d into main Nov 15, 2024
7 checks passed
@nfx nfx deleted the feat/run-as branch November 15, 2024 13:25
nfx added a commit that referenced this pull request Nov 15, 2024
* Added `make_run_as` fixture ([#82](#82)). A new pytest fixture, `make_run_as`, has been added to create an account service principal via the `acc` fixture and assign it to a workspace with default permissions, which is removed after the test is complete. The fixture creates a service principal with a random display name and assigns it to the workspace. Users can optionally assign the service principal to account groups for specific actions using the `account_groups` argument. The returned object contains properties for the workspace client, SQL backend, and SQL execution functions, as well as the display name and application ID of the ephemeral service principal. If desired, the `ws` fixture can be overridden to make all workspace fixtures provided by the plugin run as the ephemeral service principal, allowing for testing with lower privilege ephemeral service principals and improving security and isolation. This feature is not currently supported with Databricks Metadata Service authentication on Azure Databricks.
* Bump codecov/codecov-action from 4 to 5 ([#85](#85)). In this release, the Codecov GitHub Action has been updated from version 4 to 5, introducing several new features and changes. The new version uses the Codecov Wrapper to encapsulate the CLI, allowing for quicker updates to the Action. Additionally, version 5 includes an opt-out feature for tokens in public repositories, enabling contributors and other members to upload coverage reports without requiring access to the Codecov token. This can be accomplished by setting the ability for Codecov to receive a coverage report from any source in the Global Upload Token section of the settings page on codecov.io. Furthermore, the updated version introduces several new arguments, including `binary`, `gcov_args`, `gcov_executable`, `gcov_ignore`, `gcov_include`, `report_type`, and `skip_validation`, and changes the `file` and `plugin` arguments to `files` and `plugins`, respectively.
* Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 ([#80](#80)). In this release, the `databrickslabs/sandbox` dependency has been updated from version `acceptance/v0.3.1` to `0.4.2`. This update includes the addition of install instructions, more go-git libraries, and modifications to the README to explain how to use the library with the `databricks labs sandbox` command. Dependency updates include golang.org/x/crypto from version 0.16.0 to 0.17.0. The `Run nightly tests` job in the workflow has also been updated to use the new version of the `databrickslabs/sandbox/acceptance` image. The commit history for this release shows several commits, including the creation of "[TODO] XXX" issues and a full diff comparison. The pull request includes instructions for triggering Dependabot actions through comments. Reviewers are encouraged to thoroughly examine the changelog and commit history for more detailed information on the changes in this release.
* Force keyword argument in `make_query` fixture ([#81](#81)). In the latest update, the `make_query` fixture in the `redash.py` file has undergone changes to enhance code readability and maintainability. The `sql_query` parameter is now required to be passed as a keyword argument, preventing the possibility of it being mistakenly passed as a positional argument. Furthermore, the `create` function's signature has been revised to include an explicit \* before the `sql_query` parameter. It is important to note that these changes have not affected the functionality of the method, but have instead altered the parameter passing style for improved clarity. This minor update is intended to elevate the overall quality of the codebase and promote best practices.
* Renamed internal `notebooks` module to `workspace` ([#86](#86)). In this release, the internal `notebooks` module has been renamed to `workspace` to better reflect its current functionality of managing and interacting with notebooks and other resources in a Databricks workspace. This renaming applies to the import statements in `plugin.py` and `test_notebooks.py`, which has been renamed to `test_workspace.py`. Additionally, import statements for making cluster policy and instance pool permissions have been added. The functionality of the imported functions, such as `make_directory`, `make_workspace_file`, `make_notebook`, and `make_repo`, remains unchanged. This change is part of the fix for issue [#59](#59) and aims to improve the clarity and consistency of the codebase. Software engineers adopting this project should update any imports or references to the `notebooks` module to use the new `workspace` module instead.

Dependency updates:

 * Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2 ([#80](#80)).
 * Bump codecov/codecov-action from 4 to 5 ([#85](#85)).
@nfx nfx mentioned this pull request Nov 15, 2024
nfx added a commit that referenced this pull request Nov 15, 2024
* Added `make_run_as` fixture
([#82](#82)). A new
pytest fixture, `make_run_as`, has been added to create an account
service principal via the `acc` fixture and assign it to a workspace
with default permissions, which is removed after the test is complete.
The fixture creates a service principal with a random display name and
assigns it to the workspace. Users can optionally assign the service
principal to account groups for specific actions using the
`account_groups` argument. The returned object contains properties for
the workspace client, SQL backend, and SQL execution functions, as well
as the display name and application ID of the ephemeral service
principal. If desired, the `ws` fixture can be overridden to make all
workspace fixtures provided by the plugin run as the ephemeral service
principal, allowing for testing with lower privilege ephemeral service
principals and improving security and isolation. This feature is not
currently supported with Databricks Metadata Service authentication on
Azure Databricks.
* Bump codecov/codecov-action from 4 to 5
([#85](#85)). In this
release, the Codecov GitHub Action has been updated from version 4 to 5,
introducing several new features and changes. The new version uses the
Codecov Wrapper to encapsulate the CLI, allowing for quicker updates to
the Action. Additionally, version 5 includes an opt-out feature for
tokens in public repositories, enabling contributors and other members
to upload coverage reports without requiring access to the Codecov
token. This can be accomplished by setting the ability for Codecov to
receive a coverage report from any source in the Global Upload Token
section of the settings page on codecov.io. Furthermore, the updated
version introduces several new arguments, including `binary`,
`gcov_args`, `gcov_executable`, `gcov_ignore`, `gcov_include`,
`report_type`, and `skip_validation`, and changes the `file` and
`plugin` arguments to `files` and `plugins`, respectively.
* Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2
([#80](#80)). In this
release, the `databrickslabs/sandbox` dependency has been updated from
version `acceptance/v0.3.1` to `0.4.2`. This update includes the
addition of install instructions, more go-git libraries, and
modifications to the README to explain how to use the library with the
`databricks labs sandbox` command. Dependency updates include
golang.org/x/crypto from version 0.16.0 to 0.17.0. The `Run nightly
tests` job in the workflow has also been updated to use the new version
of the `databrickslabs/sandbox/acceptance` image. The commit history for
this release shows several commits, including the creation of "[TODO]
XXX" issues and a full diff comparison. The pull request includes
instructions for triggering Dependabot actions through comments.
Reviewers are encouraged to thoroughly examine the changelog and commit
history for more detailed information on the changes in this release.
* Force keyword argument in `make_query` fixture
([#81](#81)). In the
latest update, the `make_query` fixture in the `redash.py` file has
undergone changes to enhance code readability and maintainability. The
`sql_query` parameter is now required to be passed as a keyword
argument, preventing the possibility of it being mistakenly passed as a
positional argument. Furthermore, the `create` function's signature has
been revised to include an explicit \* before the `sql_query` parameter.
It is important to note that these changes have not affected the
functionality of the method, but have instead altered the parameter
passing style for improved clarity. This minor update is intended to
elevate the overall quality of the codebase and promote best practices.
* Renamed internal `notebooks` module to `workspace`
([#86](#86)). In this
release, the internal `notebooks` module has been renamed to `workspace`
to better reflect its current functionality of managing and interacting
with notebooks and other resources in a Databricks workspace. This
renaming applies to the import statements in `plugin.py` and
`test_notebooks.py`, which has been renamed to `test_workspace.py`.
Additionally, import statements for making cluster policy and instance
pool permissions have been added. The functionality of the imported
functions, such as `make_directory`, `make_workspace_file`,
`make_notebook`, and `make_repo`, remains unchanged. This change is part
of the fix for issue
[#59](#59) and aims to
improve the clarity and consistency of the codebase. Software engineers
adopting this project should update any imports or references to the
`notebooks` module to use the new `workspace` module instead.

Dependency updates:

* Bump databrickslabs/sandbox from acceptance/v0.3.1 to 0.4.2
([#80](#80)).
* Bump codecov/codecov-action from 4 to 5
([#85](#85)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant