Skip to content

Latest commit

 

History

History
239 lines (177 loc) · 8.74 KB

CONTRIBUTING.md

File metadata and controls

239 lines (177 loc) · 8.74 KB

Contributing

This document covers the following topics:

Tip

Visit Become a contributor to learn more about contributing at dbt!

The following utilities are needed for developing the packages in this repository:

  • pip
  • hatch
  • git
  • changie
  • pre-commit

Note

External contributors can contribute to this repository by forking the dbt-labs/dbt-adapters repository. For more on forking, check out the GitHub docs on forking.

Developing

This repository is a monorepo, meaning that it contains several packages that can each be installed independently. This allows for easier testing of related changes that live in different packages. For example, say you want to implement a feature in dbt-postgres that requires an update to dbt-adapters. This requires a change to at least two (dbt-tests-adapter?) different packages. This differs from a more traditional approach where each repository contains a single package. In particular, everything is moved "one level down". Make sure to take this into account when working with a specific package.

Install a package for development

Create a hatch environment

$ hatch run setup
  • Installs the package in editable mode
  • Installs development dependencies
  • Installs all repository packages in editable mode (e.g. dbt-adapters and dbt-tests-adapter)
  • Installs pre-commit hooks
  • Creates a templated test.env from test.env.example (if test.env doesn't already exist)

Tip

Configure hatch to create virtual environments in the package directory:

hatch config set dirs.env.virtual .hatch

This has two main benefits:

  1. It's easier to find the virtual environment to add it as a runner in common IDEs like VSCode and PyCharm
  2. It names the environment predictable (e.g. default, build) instead of using a hash

Activate a hatch environment

Important

Ensure that you're in a package directory prior to running any commands for a specific package, e.g.: cd dbt-postgres

$ hatch shell
(default) $

Run a command in a hatch environment

$ hatch run <command>

This is effectively shorthand for:

$ hatch shell
(default) $ <command>

Rebuild a hatch environment

# exit first if you're still in the hatch environment
(default) $ exit
$ hatch env prune
$ hatch shell

Developing against feature branches

This repository is setup to install repository packages in editable mode. For example, any local changes in dbt-adapters will be reflected in the dbt-postgres virtual environment. This will also make pull request checks work as expected without needing to alter development requirements to point to a feature branch. However, this does not work with packages outside of the repository as there is no guarantee of where they are.

Some changes require a change in dbt-common and/or dbt-core. In that case, dbt-common (or dbt-core) must be installed from that a version of branch. The best way to do this is to re-install dbt-common into your hatch environment in editable mode:

$ hatch run pip install -e ../../dbt-common

Note

Note that there are two ../ above. The first corresponds to this repository's root, and the second to your development space, e.g. ~/Source. This assumes that {username}/dbt-common is cloned alongside {username}/dbt-adapters.

Important

This command will need to be re-run every time the hatch environment is rebuilt.

You will also need to point CI to this feature branch if you are using this to test your changes. The only way to do this is to update hatch.toml to point the dependency at your feature branch:

# {adapter}/hatch.toml
[envs.default]
dependencies = [
    "dbt-common @ git+https://github.com/{username}/dbt-common.git@{branch}",
    "...",
]

Important

This update to hatch.toml will need to be reverted prior to merging into main.

Testing

Tests in this repository can be broken up into three categories:

Code quality checks

Code quality checks are configured at the repo level. These checks can run from a virtual environment or by invoking pre-commit directly:

$ hatch run code-quality

OR

$ pre-commit -run --all-files

Unit tests

Unit tests can be run locally without setting up a database connection. Unit tests need to be run from a package directory and will not work at the repo root directory.

# run all unit tests
$ hatch run unit-tests

# run all unit tests in a module
$ hatch run unit-tests tests/unit/{test_file_name}.py

# run a specific unit test
$ hatch run unit-tests tests/unit/{test_file_name}.py::{test_class_name}::{test_method_name}

Integration tests

Integration tests require setting up a database connection to run locally. This will vary by package; please refer to the package's CONTRIBUTING.md for more information.

Configure environment variables

Each adapter requires certain environment variables to connect to its platform. The template is contained in the respective test.env.exmaple file. If you already ran hatch run setup you should have a test.env file in the package root. Update the environment variables in this file with your instance's connection credentials.

Documentation

Release documentation

This repository uses changie to generate CHANGELOG entries. These entries get collated and injected into CHANGELOG.md at release, so there's no need to update CHANGELOG.md directly. Follow the steps to install changie.

Once changie is installed and the PR is created, run:

$ changie new

changie will walk through the process of creating a changelog entry. Remember to commit and push the file that's created.

Important

Do not edit the CHANGELOG.md directly. Any modifications will be lost by the consolidation process.

User documentation

Many changes will require an update to user documentation. All contributors, whether internal or external, are encouraged to open an issue and/or pull request in the docs repo when submitting user-facing changes. Here are some relevant links when considering what to update:

Submitting a pull request

Communication

Please refer to our Code of Conduct when opening issues and pull requests.

Note

We value and appreciate all contributions from our community. To ensure an organized and efficient process, we kindly ask that you refrain from tagging dbt Labs employees to request features, report bugs, or seek PR reviews. Our community guidelines also extend to all of our repositories. Please note that we have an established triage process and will respond to issues as soon as possible. If you are a dbt Cloud user, we request that you submit issues directly to our Support Team to ensure proper tracking and timely follow-up.

Signing the CLA

Important

All contributors must sign the Contributor License Agreement(CLA).

Maintainers will be unable to merge contributions until the contributor signs the CLA. This is a one time requirement, not a per-PR or per-repo requirement. Even without a CLA, anyone is welcome to open issues and comment on existing issues or pull requests.

Opening a pull request

A maintainer will be assigned to review each pull request based on priority and capacity. They may suggest code revisions for style and clarity or they may request additional tests. These are good things! dbt Labs believes that contributing high-quality code is a collaborative effort. The same process is followed whether the contributor is external or a maintainer. Once all tests are passing and the pull request has been approved by the appropriate code owners, a maintainer will merge the changes into main.

And that's it! Happy developing 🎉