created | last updated | status | reviewers | title | authors | discussion thread | |||
---|---|---|---|---|---|---|---|---|---|
2020-02-07 |
2020-09-30 |
Implemented |
|
Toolchain Transition Migration |
|
For several years, rule authors have been asking for more nuance in how targets depend on their toolchains via toolchain resolution:
- Some authors want their dependencies in the execution configuration (of the parent target)
- Some want the target configuration (again, of the parent target)
- Some want both, depending on what part of the toolchain is being built: bazelbuild/bazel#4067
This is addressed in the proposal for Toolchain Transitions, which is currently being implemented. However, the migration plan in that proposal is out of date and won't easily work: too many different rule sets are using toolchains, and there's no way to coordinate a "flag day" flip of all of them at once. Instead, it is possible to have a new migration plan for the toolchain transition, that will allow rule authors to independently migrate their rules (with some additional work). The plan is to add a new attribute during rule creation, so that individual rules can declare whether or not they are ready for the toolchain transition. There will also be an incompatible flag that overrides that attribute, so that we can assess the overall progress towards migration and begin removing the attribute.
This migration will take at least one, and probably two, major Bazel releases (so, realistically, three to six months), plus an additional release for cleanup.
Here is the proposed timeline:
- Finish implementing the toolchain transition, along with sufficient tests to be comfortable with it.
- Add a new attribute,
incompatible_use_toolchain_transition
, on both Starlark'srule
function and the nativeRuleClass.Builder
.- If the rule attribute is true, the rule will depend on toolchains via the toolchain transition.
- Add a new incompatible flag,
--incompatible_override_toolchain_transition
.- If the flag is set to true, the rule attribute is ignored and all rules use the toolchain transition.
- Wait for the above to be released.
- Rule authors migrate their rules to use toolchain transitions.
- See details below.
- Create tracking issues against rule sets to alert them of the work needed.
- When all rules covered in Bazel's CI are ready, flip the
--incompatible_override_toolchain_transition
flag to true. - Wait for release.
- Remove flag and attribute functionality, leave the
incompatible_use_toolchain_transition
rule attribute as a no-op. - Rule authors remove use of
incompatible_use_toolchain_transition
. - Wait for release.
- Remove the rule attribute entirely.
Starlark rules can migrate with the following steps:
- Add
cfg = "exec"
orcfg = "target"
to all label attributes of your toolchain rule. (ie, togo_toolchain
orscala_toolchain
)- Use the execution transition for dependencies that will execute as part of the target build.
- Use the target transition for dependencies that should be in the same
configuration as the target build.
- Technically
cfg = "target"
is not needed, as it is the default, but it is useful to document this clearly.
- Technically
- These changes are permanent.
- One possible migration path is the following:
- Add
cfg = "exec"
to every dependency. This is a no-op since currently all dependencies are in thehost
configuration. - Enable the
--incompatible_override_toolchain_transition
flag and test builds. The change should be minor, given the similarities betweenexec
andhost
dependencies, but this will reveal any cases where this isn't true. Typically, this has been seen where the dependency itself has further dependencies, which are now free to transition to unforseen configurations. Feel free to ask for assistance on the bazel-discuss mailing list or on the tracking issue for toolchain transition implementation. - Add
cfg = "target"
to dependencies where this makes sense. This change can be done individually for each dependency and tested/released separately, if desired, to ensure build correctness.
- Add
- Add
incompatible_use_toolchain_transition = True
to every rule that uses a toolchain.- That is, every rule that sets
toolchains = ["//your/rule:toolchain_type"]
. - This change will be reverted when the global migration is complete.
- That is, every rule that sets
- Test and release your rules as normal.
- When the global toolchain transition migration is complete, remove the
incompatible_use_toolchain_transition
attribute from your rules.
Note: We have seen failures with the execution transition, specifically in
cases where Android and iOS targets depend on a library which uses the execution
transition, where the configuration ends up in a confused state and the
Android/iOS flags are applied inappropriately. If this should happen, please
file an issue and ping @katre to investigate. You
can also work around this by temporarily using cfg = 'host'
for the attribute,
to simulate the current behavior, instead of cfg = 'exec'
.
Native rules migrate similarly to Starlark rules:
- Add
cfg(ExecutionTransitionFactory.create())
to the correct attributes of your toolchain rule. (ie, tocc_toolchain
orjava_toolchain
)- Use the execution transition for dependencies that will execute as part of the target build.
- Dependencies that should be in the same configuration as the target build do not need anything.
- These changes are permanent.
- Add
builder.useToolchainTransition(true)
to every rule that uses a toolchain.- That is, every rule that calls
builder.addRequiredToolchains()
. - This is inherited by child rules and so can be set at a high level.
- This change will be reverted when the global migration is complete.
- That is, every rule that calls
- Test your rules as normal, and wait for a Bazel release.
- As part of the global toolchain transition migration, your
useToolchainTransition
calls will be removed.