Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[build] All starlark externals are bazel modules #22294

Merged
merged 1 commit into from
Dec 11, 2024

Conversation

jwnimmer-tri
Copy link
Collaborator

@jwnimmer-tri jwnimmer-tri commented Dec 10, 2024

Towards #20731. Does not matter to release notes until we can announce reaching 100% on the porting.


This change is Reviewable

@jwnimmer-tri jwnimmer-tri added the release notes: none This pull request should not be mentioned in the release notes label Dec 10, 2024
@jwnimmer-tri
Copy link
Collaborator Author

+@xuchenhan-tri for both reviews per schedule (tomorrow), please.

Copy link
Contributor

@xuchenhan-tri xuchenhan-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: x2 with a question about local patches.

Reviewed 11 of 11 files at r1, all commit messages.
Reviewable status: 2 unresolved discussions


tools/workspace/workspace_bzlmod_sync_test.py line 51 at r1 (raw file):

        """Some of our repository rules have a different repository name
         compared to their module name for bzlmod. Given a module name, returns
        the expected repository name.

nit indent

Suggestion:

        """Some of our repository rules have a different repository name
         compared to their module name for bzlmod. Given a module name, returns
         the expected repository name.

tools/workspace/rules_rust/repository.bzl line 21 at r1 (raw file):

        sha256 = "ef64969a9fac996820ede477c874c15efd06d748b5b9a0372da0e64934f82989",  # noqa
        patches = [
            ":patches/import_cycle.patch",

BTW, what happens to our patches when we swtich to get these modules directly from BCR? Are they still applied? If so, where?

As of this commit, all of our starlark dependencies have been ported
from WORKSPACE to bzlmod.
Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: 1 unresolved discussion


tools/workspace/rules_rust/repository.bzl line 21 at r1 (raw file):

Previously, xuchenhan-tri wrote…

BTW, what happens to our patches when we swtich to get these modules directly from BCR? Are they still applied? If so, where?

When running in bzlmod mode, this starlark macro (def rules_rust_repository(...)) is never called, so nothing here has any effect, including this list of patches = ....

In the bigger picture, when running in bzlmod mode, any tweaks or patches to externals would only happen if MODULE.bazel says to do so, and currently our MODULE.bazel file doesn't mention any such tweaks.

In fact, customizing public externals (like starlark, eigen, etc) actually seems quite difficult with bzlmod. The guide describes several "override" functions to customize or replace the externals, but those only take effect when Drake is the main repository. When Drake is used as an external the overrides are ignored, so we are probably only going to be able to tweak things in testonly or private dependencies.

That might end up being okay, though. So far, most (all?) of our patches are to hack around things that were difficult with WORKSPACE mode, so hopefully we don't actually need much (if any) patches anymore for the public externals.

For libraries we vendor (like fcl_internal, spgrid_internal, etc.) we should be able to use patches. I believe we'll actually just call use_repo_rule to call the repository rules that we already have. The customs for our public externals vs private externals are probably going to have a much brighter distinction with bzlmod than we used to have with workspace.

Copy link
Contributor

@xuchenhan-tri xuchenhan-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r2, all commit messages.
Reviewable status: 1 unresolved discussion


tools/workspace/rules_rust/repository.bzl line 21 at r1 (raw file):
Thanks for the detailed answer. It's helpful.

That might end up being okay, though. So far, most (all?) of our patches are to hack around things that were difficult with WORKSPACE mode, so hopefully we don't actually need much (if any) patches anymore for the public externals.

What about external libraries that Drake depends on that opts into BCR? Do we have any of those? Do we expect any in the future?

Copy link
Collaborator Author

@jwnimmer-tri jwnimmer-tri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! all discussions resolved, LGTM from assignee xuchenhan-tri(platform)


tools/workspace/rules_rust/repository.bzl line 21 at r1 (raw file):

Previously, xuchenhan-tri wrote…

Thanks for the detailed answer. It's helpful.

That might end up being okay, though. So far, most (all?) of our patches are to hack around things that were difficult with WORKSPACE mode, so hopefully we don't actually need much (if any) patches anymore for the public externals.

What about external libraries that Drake depends on that opts into BCR? Do we have any of those? Do we expect any in the future?

There are already a few libraries we use which are signed up for BCR, e.g. https://registry.bazel.build/modules/gz-math. My best guess is that we'll ignore the BCR version of those. We still need the ability to vendor-bundle them (link them statically in a vendored namespace) and I don't anticipate that the BCR versions will have enough configuration options for them.

The important thing to port to use modules and BCR are our public externals (starlark rules, eigen, fmt, spdlog). Our private externals like gz_math_internal can stay private, pulled from github (not BCR), and vendored. Eventually we might offer further customization options for those depending on user demands, but not right away.

@jwnimmer-tri jwnimmer-tri merged commit 9e6a49e into RobotLocomotion:master Dec 11, 2024
9 of 10 checks passed
@jwnimmer-tri jwnimmer-tri deleted the bzlmod-starlark branch December 11, 2024 02:20
RussTedrake pushed a commit to RussTedrake/drake that referenced this pull request Dec 15, 2024
As of this commit, all of our starlark dependencies have been ported
from WORKSPACE to bzlmod.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: medium release notes: none This pull request should not be mentioned in the release notes status: single reviewer ok https://drake.mit.edu/reviewable.html
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants