Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cc-wrapper, bintools-wrapper: Improve hooks that run before invocation #366960

Merged
merged 2 commits into from
Dec 23, 2024

Conversation

pwaller
Copy link
Contributor

@pwaller pwaller commented Dec 20, 2024

RFC: Improve these hooks. bintools only had a hook which could run after the linker, which meant it could not be used to modify arguments. cc-wrapper-hook could run before the compiler invocation, but it ran before the NIX_DEBUG, which implied that if the wrapper were to modify arguments, it would not be reflected in the debug output.

  • bintools-wrapper: introduce ld-wrapper-hook
  • cc-wrapper: Move cc-wrapper-hook above NIX_DEBUG

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

The existing post-link-hook is in an unfortunate position since it does
not enable interposing the arguments of the compiler.

This is an RFC to add a hook which enables that.

Signed-off-by: Peter Waller <[email protected]>
I observe that the old position meant that if the wrapper were to
interfere with the arguments going into the compiler, it would not be
reflected in the debug output.

Signed-off-by: Peter Waller <[email protected]>
@github-actions github-actions bot added the 8.has: documentation This PR adds or changes documentation label Dec 20, 2024
@pwaller
Copy link
Contributor Author

pwaller commented Dec 20, 2024

/cc @Ericson2314 in case you have any opinions. I have not yet had a chance to use these, but I have use cases for them for experimentation and development. I hope they are obvious source modifications. Being able to directly influence extraBefore and extraAfter in particular after all other processing is a useful capability.

@pwaller
Copy link
Contributor Author

pwaller commented Dec 21, 2024

Test failure is that the manual failed to build because it had to build everything from scratch and looks like it timed out after 6 hours.

@pwaller pwaller marked this pull request as ready for review December 21, 2024 10:44
@nix-owners nix-owners bot requested a review from Ericson2314 December 21, 2024 10:46
@ofborg ofborg bot added 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild 10.rebuild-linux-stdenv This PR causes stdenv to rebuild 10.rebuild-darwin: 501+ 10.rebuild-linux: 501+ labels Dec 21, 2024
@pwaller
Copy link
Contributor Author

pwaller commented Dec 23, 2024

/cc @RossComputerGuy @philiptaron @emilazy (hope you don't mind the ping). I think this change is an obvious small source transformation, though it incurs a stdenv rebuild so has not received much testing.

Slightly unfortunate to end up with two hooks for the linker wrapper; I opted not to remove the old one to retain backwards compatibility for now.

Copy link
Contributor

@philiptaron philiptaron left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change to run before the logging makes a lot of sense to me.

The additional hook also makes sense to me, since it allows for mutation of the linker arguments. Doing the search on post-link-hook and seeing that it's used for codesigning on Darwin also clarifies its usecase and lets me feel assured that supporting two hooks is a worthwhile idea.

@@ -1411,6 +1411,7 @@ these in the [Hooks Reference](#chap-hooks).
### Compiler and Linker wrapper hooks {#compiler-linker-wrapper-hooks}

If the file `${cc}/nix-support/cc-wrapper-hook` exists, it will be run at the end of the [compiler wrapper](#cc-wrapper).
If the file `${binutils}/nix-support/ld-wrapper-hook` exists, it will be run at the end of the linker wrapper, before the linker runs.
If the file `${binutils}/nix-support/post-link-hook` exists, it will be run at the end of the linker wrapper.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Within nixpkgs, this appears to be used by pkgs/os-specific/darwin/signing-utils/post-link-sign-hook.nix only.

{
writeTextFile,
cctools,
sigtool,
}:
writeTextFile {
name = "post-link-sign-hook";
executable = true;
text = ''
if [ "$linkerOutput" != "/dev/null" ]; then
CODESIGN_ALLOCATE=${cctools}/bin/${cctools.targetPrefix}codesign_allocate \
${sigtool}/bin/codesign -f -s - "$linkerOutput"
fi
'';
}

Copy link
Contributor

@reckenrode reckenrode Dec 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This hook is nearly vestigial. Darwin only uses this hook early in the bootstrap. Once it builds a version of ld64 supporting code-sigining, it’s no longer used. It can be dropped once Darwin’s bootstrap tools are updated with that version of ld64, which should hopefully be doable after this staging-next merge. Getting clang 19 into the bootstrap is also important for other reasons (such as guaranteeing C23 support in the bootstrap).

Edit: Looks like staging-next was merged, so now just the bootstrap tools need updated.

@philiptaron
Copy link
Contributor

I'm going to run the stdenv test suite then I plan on merging this. Please countermand me, @NixOS/stdenv, if you disagree!

@philiptaron philiptaron merged commit 558c04a into NixOS:staging Dec 23, 2024
60 of 61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants