Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add no-default-feature option #1092

Merged
merged 33 commits into from
Apr 29, 2024
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
6eae2b7
Add no-default-feature option
olivier-lacroix Mar 31, 2024
5a578b8
Add test
olivier-lacroix Mar 31, 2024
1801aa5
add to polarify example
olivier-lacroix Apr 1, 2024
63c63ef
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 2, 2024
eb22b4f
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 2, 2024
9169083
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 2, 2024
8bc062c
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 3, 2024
3c6d447
Update docs/configuration.md
olivier-lacroix Apr 3, 2024
aec2a34
fix: polarify example and add a warning.
ruben-arts Apr 3, 2024
cee5942
Merge branch 'main' into nodefaultfeature
tdejager Apr 5, 2024
b015263
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 6, 2024
d85d9af
Start refactoring
olivier-lacroix Apr 6, 2024
6d27c45
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 20, 2024
c9a3dc5
handle defautl inclusion in solve groups
olivier-lacroix Apr 20, 2024
2b11c94
Handle default feature inclusion in GroupedEnvironement
olivier-lacroix Apr 20, 2024
ce3d63e
revert unrelated change
olivier-lacroix Apr 20, 2024
ff03d59
Implement deserialisation logic and tests
olivier-lacroix Apr 20, 2024
d42171e
Update syntax in examples, tests and documentation
olivier-lacroix Apr 20, 2024
dc18fe1
Regenerate schema
olivier-lacroix Apr 20, 2024
271e5ea
switch to a platform selector
olivier-lacroix Apr 20, 2024
9c99598
Improve documentation
olivier-lacroix Apr 20, 2024
e0a7bae
Refactor to leverage Environment methods in SolveGroup
olivier-lacroix Apr 20, 2024
9c51c3a
Rename
olivier-lacroix Apr 21, 2024
86e7789
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 26, 2024
195ae29
Fix resolution of channels for environments
olivier-lacroix Apr 26, 2024
7661d8a
Merge branch 'main' into nodefaultfeature
olivier-lacroix Apr 27, 2024
0414ac2
Revert to boolean no-default-feature
olivier-lacroix Apr 27, 2024
7f93887
Improve documentation
olivier-lacroix Apr 27, 2024
04cdb23
Refactor commonalities between environment, solvegroup and groupedenv…
olivier-lacroix Apr 28, 2024
29b1a6b
feat: change trait name to `HasFeatures`
tdejager Apr 29, 2024
1616205
feat: renamed module
tdejager Apr 29, 2024
550d516
misc: use no-default-feature in pixi itself and add to the docs example
ruben-arts Apr 29, 2024
6d75bf9
fix: revert use of no-default in pixi as this needs to be merged firs…
ruben-arts Apr 29, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 3 additions & 6 deletions docs/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -539,18 +539,15 @@ platforms = ["linux-64", "osx-arm64"]
### The `environments` table
The `environments` table allows you to define environments that are created using the features defined in the `feature` tables.

!!! important
`default` is always implied when creating environments.
If you don't want to use the `default` feature you can keep all the non feature tables empty.

The environments table is defined using the following fields:

- `features: Vec<Feature>`: The features that are included in the environment set, which is also the default field in the environments.
- `solve-group: String`: The solve group is used to group environments together at the solve stage.
- `features`: The features that are included in the environment, which is also the default field in the environments. Unless `no-default-feature` is set to `false`; the default feature is always included.
olivier-lacroix marked this conversation as resolved.
Show resolved Hide resolved
- `solve-group`: The solve group is used to group environments together at the solve stage.
This is useful for environments that need to have the same dependencies but might extend them with additional dependencies.
For instance when testing a production environment with additional test dependencies.
These dependencies will then be the same version in all environments that have the same solve group.
But the different environments contain different subsets of the solve-groups dependencies set.
- `no-default-feature`: Whether to include the default feature in that environment. The default is to include the default feature.

```toml title="Simplest example"
[environments]
Expand Down
2 changes: 1 addition & 1 deletion examples/polarify/pixi.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ py39 = ["py39", "test"]
py310 = ["py310", "test"]
py311 = ["py311", "test"]
py312 = ["py312", "test"]
lint = ["lint"]
lint = { features=["lint"], no-default-feature=true}
olivier-lacroix marked this conversation as resolved.
Show resolved Hide resolved

## test this with:
#pixi run test
Expand Down
10 changes: 8 additions & 2 deletions schema/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,9 @@ class PyPIPathRequirement(_PyPIRequirement):
None,
description="A path to a local source or wheel",
)
editable: Optional[bool] = Field(None, description="If true the package will be installed as editable")

editable: Optional[bool] = Field(
None, description="If true the package will be installed as editable"
)


class PyPIUrlRequirement(_PyPIRequirement):
Expand Down Expand Up @@ -254,6 +255,11 @@ class Environment(StrictBaseModel):
alias="solve-group",
description="The group name for environments that should be solved together",
)
no_default_feature: Optional[bool] = Field(
False,
alias="no-default-feature",
description="Whether to add the default feature automatically",
)


######################
Expand Down
13 changes: 13 additions & 0 deletions schema/schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,19 @@
"default": null,
"description": "The group name for environments that should be solved together",
"title": "Solve-Group"
},
"no-default-feature": {
"anyOf": [
{
"type": "boolean"
},
{
"type": "null"
}
],
"default": false,
"description": "Whether to add the default feature automatically",
"title": "No-Default-Feature"
}
},
"title": "Environment",
Expand Down
7 changes: 2 additions & 5 deletions src/cli/info.rs
Original file line number Diff line number Diff line change
Expand Up @@ -307,17 +307,14 @@ pub async fn execute(args: Args) -> miette::Result<()> {
.iter()
.map(|env| {
let tasks = env
.tasks(None, true)
.tasks(None)
.ok()
.map(|t| t.into_keys().cloned().collect())
.unwrap_or_default();

EnvironmentInfo {
name: env.name().clone(),
features: env
.features(true)
.map(|feature| feature.name.clone())
.collect(),
features: env.features().map(|feature| feature.name.clone()).collect(),
solve_group: env
.solve_group()
.map(|solve_group| solve_group.name().to_string()),
Expand Down
4 changes: 2 additions & 2 deletions src/cli/run.rs
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ fn command_not_found<'p>(project: &'p Project, explicit_environment: Option<Envi
let available_tasks: HashSet<TaskName> =
if let Some(explicit_environment) = explicit_environment {
explicit_environment
.tasks(Some(Platform::current()), true)
.tasks(Some(Platform::current()))
.into_iter()
.flat_map(|tasks| tasks.into_keys())
.map(ToOwned::to_owned)
Expand All @@ -208,7 +208,7 @@ fn command_not_found<'p>(project: &'p Project, explicit_environment: Option<Envi
.into_iter()
.filter(|env| verify_current_platform_has_required_virtual_packages(env).is_ok())
.flat_map(|env| {
env.tasks(Some(Platform::current()), true)
env.tasks(Some(Platform::current()))
.into_iter()
.flat_map(|tasks| tasks.into_keys())
.map(ToOwned::to_owned)
Expand Down
2 changes: 1 addition & 1 deletion src/cli/task.rs
Original file line number Diff line number Diff line change
Expand Up @@ -257,7 +257,7 @@ pub fn execute(args: Args) -> miette::Result<()> {
let tasks = project
.environment(&env)
.ok_or(miette!("Environment `{}` not found in project", env))?
.tasks(Some(Platform::current()), true)?
.tasks(Some(Platform::current()))?
.into_keys()
.collect_vec();
if tasks.is_empty() {
Expand Down
38 changes: 15 additions & 23 deletions src/project/environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -106,12 +106,8 @@ impl<'p> Environment<'p> {
.join(self.environment.name.as_str())
}

/// Returns references to the features that make up this environment. The default feature is
/// always added at the end.
pub fn features(
&self,
include_default: bool,
) -> impl DoubleEndedIterator<Item = &'p Feature> + 'p {
/// Returns references to the features that make up this environment.
pub fn features(&self) -> impl DoubleEndedIterator<Item = &'p Feature> + 'p {
let environment_features = self.environment.features.iter().map(|feature_name| {
self.project
.manifest
Expand All @@ -121,10 +117,10 @@ impl<'p> Environment<'p> {
.expect("feature usage should have been validated upfront")
});

if include_default {
Either::Left(environment_features.chain([self.project.manifest.default_feature()]))
} else {
if self.environment.no_default_feature {
Either::Right(environment_features)
} else {
Either::Left(environment_features.chain([self.project.manifest.default_feature()]))
}
}

Expand All @@ -138,7 +134,7 @@ impl<'p> Environment<'p> {
/// used instead. However, these are not considered during deduplication. This means the default
/// channels are always added to the end of the list.
pub fn channels(&self) -> IndexSet<&'p Channel> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation is not aligned with the code here. currently there is no deduplication, and a name feature will never use the default channels. what should it be?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The true in self.features(true) includes the default feature.

The function returns an IndexSet which deduplicates the channels.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes @baszalmstra. But default channels are also deduplicated though?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mm, Im afraid I dont quite understand what you mean?

Copy link
Contributor Author

@olivier-lacroix olivier-lacroix Apr 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See new proposed implementation @baszalmstra . I believe it keeps the project default channels at the back, but works whether or not the default feature is included in the environment. Let me know what you think!

self.features(true)
self.features()
.filter_map(|feature| match feature.name {
// Use the user-specified channels of each feature if the feature defines them. Only
// for the default feature do we use the default channels from the project metadata
Expand Down Expand Up @@ -171,7 +167,7 @@ impl<'p> Environment<'p> {
/// Features can specify which platforms they support through the `platforms` key. If a feature
/// does not specify any platforms the features defined by the project are used.
pub fn platforms(&self) -> HashSet<Platform> {
self.features(true)
self.features()
.map(|feature| {
match &feature.platforms {
Some(platforms) => &platforms.value,
Expand All @@ -196,11 +192,10 @@ impl<'p> Environment<'p> {
pub fn tasks(
&self,
platform: Option<Platform>,
include_default: bool,
) -> Result<HashMap<&'p TaskName, &'p Task>, UnsupportedPlatformError> {
self.validate_platform_support(platform)?;
let result = self
.features(include_default)
.features()
.flat_map(|feature| feature.targets.resolve(platform))
.rev() // Reverse to get the most specific targets last.
.flat_map(|target| target.tasks.iter())
Expand All @@ -215,10 +210,7 @@ impl<'p> Environment<'p> {
name: &TaskName,
platform: Option<Platform>,
) -> Result<&'p Task, UnknownTask> {
match self
.tasks(platform, true)
.map(|tasks| tasks.get(name).copied())
{
match self.tasks(platform).map(|tasks| tasks.get(name).copied()) {
Err(_) | Ok(None) => Err(UnknownTask {
project: self.project,
environment: self.name().clone(),
Expand Down Expand Up @@ -259,7 +251,7 @@ impl<'p> Environment<'p> {
/// the features that make up the environment. If multiple features specify a requirement for
/// the same system package, the highest is chosen.
pub fn local_system_requirements(&self) -> SystemRequirements {
self.features(true)
self.features()
.map(|feature| &feature.system_requirements)
.fold(SystemRequirements::default(), |acc, req| {
acc.union(req)
Expand All @@ -273,7 +265,7 @@ impl<'p> Environment<'p> {
/// requirement for the same package that both requirements are returned. The different
/// requirements per package are sorted in the same order as the features they came from.
pub fn dependencies(&self, kind: Option<SpecType>, platform: Option<Platform>) -> Dependencies {
self.features(true)
self.features()
.filter_map(|f| f.dependencies(kind, platform))
.map(|deps| Dependencies::from(deps.into_owned()))
.reduce(|acc, deps| acc.union(&deps))
Expand All @@ -289,7 +281,7 @@ impl<'p> Environment<'p> {
&self,
platform: Option<Platform>,
) -> IndexMap<PyPiPackageName, Vec<PyPiRequirement>> {
self.features(true)
self.features()
.filter_map(|f| f.pypi_dependencies(platform))
.fold(IndexMap::default(), |mut acc, deps| {
// Either clone the values from the Cow or move the values from the owned map.
Expand All @@ -316,7 +308,7 @@ impl<'p> Environment<'p> {
/// The activation scripts of all features are combined in the order they are defined for the
/// environment.
pub fn activation_scripts(&self, platform: Option<Platform>) -> Vec<String> {
self.features(true)
self.features()
.filter_map(|f| f.activation_scripts(platform))
.flatten()
.cloned()
Expand All @@ -343,7 +335,7 @@ impl<'p> Environment<'p> {

/// Returns true if the environments contains any reference to a pypi dependency.
pub fn has_pypi_dependencies(&self) -> bool {
self.features(true).any(|f| f.has_pypi_dependencies())
self.features().any(|f| f.has_pypi_dependencies())
}
}

Expand Down Expand Up @@ -449,7 +441,7 @@ mod tests {

assert!(manifest
.default_environment()
.tasks(Some(Platform::Osx64), true)
.tasks(Some(Platform::Osx64))
.is_err())
}

Expand Down
4 changes: 2 additions & 2 deletions src/project/grouped_environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ impl<'p> GroupedEnvironment<'p> {
/// Returns the features of the group
pub fn features(&self) -> impl DoubleEndedIterator<Item = &'p Feature> + 'p {
olivier-lacroix marked this conversation as resolved.
Show resolved Hide resolved
match self {
GroupedEnvironment::Group(group) => Either::Left(group.features(true)),
GroupedEnvironment::Environment(env) => Either::Right(env.features(true)),
GroupedEnvironment::Group(group) => Either::Left(group.features()),
GroupedEnvironment::Environment(env) => Either::Right(env.features()),
}
}
}
Expand Down
5 changes: 5 additions & 0 deletions src/project/manifest/environment.rs
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,9 @@ pub struct Environment {
/// An optional solver-group. Multiple environments can share the same solve-group. All the
/// dependencies of the environment that share the same solve-group will be solved together.
pub solve_group: Option<usize>,

/// Whether to include the default feature automatically or not
pub no_default_feature: bool,
}

/// Helper struct to deserialize the environment from TOML.
Expand All @@ -134,6 +137,8 @@ pub(super) struct TomlEnvironment {
#[serde(default)]
pub features: PixiSpanned<Vec<String>>,
pub solve_group: Option<String>,
#[serde(default)]
pub no_default_feature: bool,
}

pub(super) enum TomlEnvironmentMapOrSeq {
Expand Down
15 changes: 10 additions & 5 deletions src/project/manifest/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1029,6 +1029,7 @@ impl<'de> Deserialize<'de> for ProjectManifest {
features: Vec::new(),
features_source_loc: None,
solve_group: None,
no_default_feature: false,
});
environments.by_name.insert(EnvironmentName::Default, 0);
}
Expand All @@ -1039,11 +1040,14 @@ impl<'de> Deserialize<'de> for ProjectManifest {
environments.by_name.insert(name.clone(), environment_idx);

// Decompose the TOML
let (features, features_source_loc, solve_group) = match env {
TomlEnvironmentMapOrSeq::Map(env) => {
(env.features.value, env.features.span, env.solve_group)
}
TomlEnvironmentMapOrSeq::Seq(features) => (features, None, None),
let (features, features_source_loc, solve_group, no_default_feature) = match env {
TomlEnvironmentMapOrSeq::Map(env) => (
env.features.value,
env.features.span,
env.solve_group,
env.no_default_feature,
),
TomlEnvironmentMapOrSeq::Seq(features) => (features, None, None, false),
};

// Add to the solve group if defined
Expand Down Expand Up @@ -1074,6 +1078,7 @@ impl<'de> Deserialize<'de> for ProjectManifest {
features,
features_source_loc,
solve_group,
no_default_feature,
});
}

Expand Down
2 changes: 1 addition & 1 deletion src/project/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -352,7 +352,7 @@ impl Project {
/// TODO: Remove this function and use the tasks from the default environment instead.
pub fn tasks(&self, platform: Option<Platform>) -> HashMap<&TaskName, &Task> {
self.default_environment()
.tasks(platform, true)
.tasks(platform)
.unwrap_or_default()
}

Expand Down
Loading
Loading