Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build-support: add new hooks #370869

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

drupol
Copy link
Contributor

@drupol drupol commented Jan 4, 2025

Hello everyone,

During the holidays, I spent some time packaging Python packages and noticed that some of them require a valid and mutable home directory to build or function properly.

Currently, the default behavior of the Nix stdenv is to set the HOME environment variable to /homeless-shelter, a read-only directory. While this approach is consistent, it can cause issues for packages that expect a writable home directory.

After performing a treewide search in nixpkgs, I found approximately 500 instances where export HOME is used (rg -c "export HOME" | wc -l) and 300 where export PATH is used (rg -c "export PATH" | wc -l). These repetitive patterns inspired me to propose a solution adhering to the DRY principle.

I created custom hooks, tmpdirAsHomeHook, which sets up a mutable home directory automatically when added to nativeBuildInputs or another appropriate place. Additionally, addBinToPath, which modifies the PATH environment variable to include bin/ if available.

I’d appreciate your feedback on this approach and have a few questions:

  1. Do you think this would be a valuable addition to nixpkgs?
  2. Regarding naming, I’m considering renaming tmpdirAsHomeHook to mutableHomeDirHook for better clarity. What do you think?
  3. In the hooks, I use the line addEnvHooks "$targetOffset" addBinToPath. I’m not entirely confident about the best way to implement this. Could you provide some guidance?

Thanks for taking the time to review this. I look forward to hearing your thoughts and suggestions!

Best regards

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@drupol drupol changed the title treewide: add two custom hooks build-support: add two custom hooks Jan 4, 2025
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/request-for-comments-adding-two-new-hooks/58282/1

@drupol drupol mentioned this pull request Jan 9, 2025
13 tasks
@drupol drupol changed the title build-support: add two custom hooks build-support: add new hook mutableHomeDirHook Jan 9, 2025
@drupol drupol changed the title build-support: add new hook mutableHomeDirHook build-support: add new hook tmpdirAsHome Jan 9, 2025
@drupol drupol force-pushed the push-zyrxtzozkqnt branch 3 times, most recently from 9e45c4c to c30ae4a Compare January 11, 2025 21:12
@github-actions github-actions bot added the 8.has: documentation This PR adds or changes documentation label Jan 11, 2025
@drupol drupol force-pushed the push-zyrxtzozkqnt branch from c30ae4a to 1a6ea00 Compare January 11, 2025 21:26
@drupol drupol marked this pull request as ready for review January 11, 2025 21:30
@drupol drupol changed the title build-support: add new hook tmpdirAsHome build-support: add new hooks Jan 11, 2025
@drupol drupol force-pushed the push-zyrxtzozkqnt branch from 1a6ea00 to 48ddab5 Compare January 13, 2025 14:52
@drupol drupol force-pushed the push-zyrxtzozkqnt branch from 48ddab5 to 2246ddf Compare January 13, 2025 15:27
@drupol drupol force-pushed the push-zyrxtzozkqnt branch from 2246ddf to e45fb8f Compare January 13, 2025 15:32
@xiaoxiangmoe
Copy link
Contributor

xiaoxiangmoe commented Jan 13, 2025

Will this hook also be avaliable in preConfig or some thing like this?

Here is some example:

export HOME="$NIX_BUILD_TOP"

export HOME="$NIX_BUILD_TOP"

@drupol
Copy link
Contributor Author

drupol commented Jan 13, 2025

Will this hook also be avaliable in preConfig or some thing like this?

Here is some example:

export HOME="$NIX_BUILD_TOP"

export HOME="$NIX_BUILD_TOP"

It can actually be added to any phase hooks, so yeah.

Copy link
Contributor

@hsjobeki hsjobeki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the Idea overall.
I also have two feedback Points:

  • This could be two PRs.
  • For my taste the amount of documentation is not enough. Especially i wouldn like to know stuff that happens internally, like what value is. $HOME set to and when?

@drupol
Copy link
Contributor Author

drupol commented Jan 14, 2025

  • This could be two PRs.

While I'm all for splitting PR, make sure that they have a clear scope, I really don't understand why I would need to split this one. The scope is clear, even the commits are correctly split. So I'm sorry but no, I won't split this PR in two.

  • For my taste the amount of documentation is not enough. Especially i wouldn like to know stuff that happens internally, like what value is. $HOME set to and when?

OK I will improve it.

@drupol drupol force-pushed the push-zyrxtzozkqnt branch from e45fb8f to e8e0a8d Compare January 14, 2025 07:27
@drupol drupol mentioned this pull request Jan 26, 2025
12 tasks
@drupol drupol force-pushed the push-zyrxtzozkqnt branch 2 times, most recently from 4823191 to 881d72c Compare January 26, 2025 18:20
@drupol
Copy link
Contributor Author

drupol commented Jan 26, 2025

Planning to merge this tomorrow morning, feel free to raise your voice if you believe it is a bad idea.

@K900
Copy link
Contributor

K900 commented Jan 26, 2025

I'd like to see some examples of packages helped by these hooks, at the very least.

@drupol
Copy link
Contributor Author

drupol commented Jan 26, 2025

I'd like to see some examples of packages helped by these hooks, at the very least.

Here's some Github search where it would be used:

@@ -1375,6 +1375,26 @@ This hook only runs when compiling for Linux.

This sets `SOURCE_DATE_EPOCH` to the modification time of the most recent file.

### `add-bin-to-path.sh` {#add-bin-to-path.sh}

This setup hook adds the `./bin/` directory to the `PATH` environment variable,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably at least explicitly mention it's adding $out/bin and not $sourceRoot/bin or whatever. Honestly, this is my big concern with this hook: it's not immediately obvious what exactly it does, and the thing it does comes with a bunch of weird caveats (what if there are multiple outputs? what if the binaries are in sbin? etc).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you, I just updated the documentation. Let me know if this resolves this feedback.

@drupol drupol force-pushed the push-zyrxtzozkqnt branch from 881d72c to f811073 Compare January 27, 2025 18:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants