Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Experimental portable lockdirs #11489

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

gridbugs
Copy link
Collaborator

Adds a feature flag for enabling portable lockdirs. This is a proof of concept implementation of portable lockdirs where the entire solver runs for each of a set of platforms (combinations of architecture, OS, and in some cases the OS distribution) which most people are expected to use. This can easily be extended in the future to add more platforms or to allow projects to specify more platforms.

To make lockdirs portable, the build/install commands and dependencies of each package are transformed into match statements, where the appropriate value for each platform is enumerated. At solve-time, the solver runs once for each platform, populating these fields. At build-time, the command/dependencies appropriate for the current platform are used.

When the feature flag is not enabled dune's behaviour is unchanged.

Related to #11476

@gridbugs
Copy link
Collaborator Author

See https://github.com/ocaml/dune/pull/11489/files#diff-2d6698a9af755d43a10093ee8248c2f6c9606358252ad453483447bf0d302fc5 for an example of how lockfiles look with this feature enabled.

@gridbugs gridbugs force-pushed the portable-lockdir-feature branch 4 times, most recently from 59d5fe2 to beebd21 Compare February 24, 2025 04:25
@gridbugs
Copy link
Collaborator Author

The failing lock-directory-selection test on macos is due to the issue fixed in #11497.

Adds a feature flag for enabling portable lockdirs. This is a proof of
concept implementation of portable lockdirs where the entire solver runs
for each of a set of platforms (combinations of architecture, OS, and in
some cases the OS distribution) which most people are expected to use.
This can easily be extended in the future to add more platforms or to
allow projects to specify more platforms.

To make lockdirs portable, the build/install commands and dependencies
of each package are transformed into match statements, where the
appropriate value for each platform is enumerated. At solve-time, the
solver runs once for each platform, populating these fields. At
build-time, the command/dependencies appropriate for the current
platform are used.

When the feature flag is not enabled dune's behaviour is unchanged.

Signed-off-by: Stephen Sherratt <[email protected]>
@gridbugs gridbugs force-pushed the portable-lockdir-feature branch from 5319f22 to fb43b79 Compare March 3, 2025 06:09
@gridbugs
Copy link
Collaborator Author

gridbugs commented Mar 3, 2025

@rgrinberg can you take a look at this please

@rgrinberg
Copy link
Member

I'll try and get around to this in the next few days. Meanwhile can @art-w or @Leonidas-from-XIV give this a review?

@ManasJayanth
Copy link
Contributor

Is Dune_pkg.Solver_env.popular_platform_envs configurable? I recently implemented this feature for esy at esy/esy#1640 - one difference being the list of environments is configurable so we dont spend time solving for platforms user doesn't want us to.

@gridbugs
Copy link
Collaborator Author

Is Dune_pkg.Solver_env.popular_platform_envs configurable?

Not yet, but that's something I'm thinking of adding in the future. The goal for this PR is to show a proof of concept for portable lockdirs but there's several extensions I want to make to improve usability of this feature once the current PR is merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants