doc: add chapter Fixed-point arguments of build helpers

Add "Fixed-point arguments of build helpers" chapter in "Builde helpers" part. Co-authored-by: nicoo <[email protected]> Co-authored-by: Silvan Mosberger <[email protected]> Co-authored-by: Valentin Gagarin <[email protected]> Co-authored-by: Lin Jian <[email protected]> Co-authored-by: Philip Taron <[email protected]>
NixOS · Jan 15, 2025 · 83c0e00 · 83c0e00
1 parent 2b52002
commit 83c0e00
Show file tree

Hide file tree

Showing 3 changed files with 221 additions and 0 deletions.
diff --git a/doc/build-helpers.md b/doc/build-helpers.md
@@ -17,6 +17,7 @@ There is no uniform interface for build helpers.
 [Language- or framework-specific build helpers](#chap-language-support) usually follow the style of `stdenv.mkDerivation`, which accepts an attribute set or a fixed-point function taking an attribute set.
 
 ```{=include=} chapters
+build-helpers/fixed-point-arguments.chapter.md
 build-helpers/fetchers.chapter.md
 build-helpers/trivial-build-helpers.chapter.md
 build-helpers/testers.chapter.md

diff --git a/doc/build-helpers/fixed-point-arguments.chapter.md b/doc/build-helpers/fixed-point-arguments.chapter.md
@@ -0,0 +1,202 @@
+# Fixed-point arguments of build helpers {#chap-build-helpers-finalAttrs}
+
+As mentioned in the beginning of this part, `stdenv.mkDerivation` could alternatively accept a fixed-point function. The input of such function, typically named `finalAttrs`, is expected to be the final state of the attribute set.
+A build helper like this is said to accept **fixed-point arguments**.
+
+Build helpers don't always support fixed-point arguments yet, as support in [`stdenv.mkDerivation`](#mkderivation-recursive-attributes) was first included in Nixpkgs 22.05.
+
+## Defining a build helper with `lib.extendMkDerivation` {#sec-build-helper-extendMkDerivation}
+
+Developers can use the Nixpkgs library function [`lib.customisation.extendMkDerivation`](#function-library-lib.customisation.extendMkDerivation) to define a build helper supporting fixed-point arguments from an existing one with such support, with an attribute overlay similar to the one taken by [`<pkg>.overrideAttrs`](#sec-pkg-overrideAttrs).
+
+:::{.example #ex-build-helpers-extendMkDerivation-simple}
+
+# Example definition of `mkLocalDerivation` extended from `stdenv.mkDerivation` with `lib.extendMkDerivation`
+
+We want to define a build helper named `mkLocalDerivation` that builds locally without using substitutes by default.
+
+Instead of taking a plain attribute set,
+
+```nix
+{
+  preferLocalBuild ? true,
+  allowSubstitute ? false,
+  ...
+}@args:
+
+stdenv.mkDerivation (
+  args
+  // {
+    # Attributes to update
+    inherit
+      preferLocalBuild
+      allowSubstitute
+      ;
+  }
+)
+```
+
+we could define with `lib.extendMkDerivation` an attribute overlay to make the result build helper also accepts the the attribute set's fixed point passing to the underlying `stdenv.mkDerivation`, named `finalAttrs` here:
+
+```nix
+lib.extendMkDerivation {
+  constructDrv = stdenv.mkDerivation;
+  extendArgs =
+    finalAttrs:
+    {
+      preferLocalBuild ? true,
+      allowSubstitute ? false,
+      ...
+    }@args:
+
+    # No need of `args //` here.
+    # The whole set of input arguments is passed in advance.
+    {
+      # Attributes to update
+      inherit
+        preferLocalBuild
+        allowSubstitute
+        ;
+    };
+}
+```
+:::
+
+Should there be a need to apply extra changes to the result derivation, pass the derivation transformation function to `lib.extendMkDerivation` as `lib.customisation.extendMkDerivation { transformDrv = drv: ...; }`.
+
+### Optionally remove specified arguments with `lib.extendMkDerivation` {#sec-build-helper-extendMkDerivation-removedArgNames}
+
+Many existing build helpers only pass part of their arguments down to their base build helper, and developers may also remove some arguments when implementing compatibility layers. To address that, `lib.extendMkDerivation` optionally takes a list of argument names to remove (`removedArgNames`).
+
+:::{.example #ex-build-helpers-extendMkDerivation-removedArgNames}
+
+# Example definition of `mkLocalDerivation` with `lib.extendMkDerivation` and `removedArgNames` specified
+
+Let's take as our example a build helper `mkLocalDerivation`, which requires an argument `specialArg` that cannot be passed to `stdenv.mkDerivation`.
+
+```nix
+{
+  preferLocalBuild ? true,
+  allowSubstitute ? false,
+  specialArg ? (_: false),
+  ...
+}@args:
+
+stdenv.mkDerivation (
+  removeAttrs [
+    # Don't pass specialArg into mkDerivation.
+    "specialArg"
+  ] args
+  // {
+    # Arguments to pass
+    inherit preferLocalBuild allowSubstitute;
+    # Some expressions involving specialArg
+    greeting = if specialArg "hi" then "hi" else "hello";
+  }
+)
+```
+
+Specify `removedArgNames` attribute so that `specialArg` won't be passed to `stdenv.mkDerivation`.
+
+```nix
+lib.extendMkDerivation {
+  constructDrv = stdenv.mkDerivation;
+  extendArgs =
+    finalAttrs:
+    {
+      preferLocalBuild ? true,
+      allowSubstitute ? false,
+      specialArg ? (_: false),
+      ...
+    }@args:
+    {
+      # Arguments to pass
+      inherit
+        preferLocalBuild
+        allowSubstitute
+        ;
+      # Some expressions involving specialArg
+      greeting = if specialArg "hi" then "hi" else "hello";
+    };
+  removedArgNames = [
+    # Don't pass specialArg into mkDerivation.
+    "specialArg"
+  ];
+}
+```
+:::
+
+Nevertheless, those removed arguments are often hard to override via `<pkg>.overrideAttrs`, leading to the use of custom overriders (such as `overridePythonPackage`) and extra complexity when overriding. In the long run, we would like to refactor build helpers to pass every argument down to `stdenv.mkDerivation`, so that they can all be overridden by [`overrideAttrs`](#sec-pkg-overrideAttrs), eliminating the use of custom overriders (e.g., `overridePythonAttrs`).
+
+The following example shows a smooth migration that empties the `removedArgNames` by guiding users to pass `specialArg` via `passthru`:
+
+:::{.example #ex-build-helpers-extendMkDerivation-removedArgNames-migration}
+
+# Getting rid of `removedArgNames` in a backport-compatible way
+
+Refactor the definition to pass `specialArg` properly while keeping some backward compatibility.
+
+```nix
+lib.extendMkDerivation {
+  constructDrv = stdenv.mkDerivation;
+  extendArgs =
+    finalAttrs:
+    {
+      preferLocalBuild ? true,
+      allowSubstitute ? false,
+      ...
+    }@args:
+    {
+      # Arguments to pass
+      inherit
+        preferLocalBuild
+        allowSubstitute
+        ;
+      passthru = {
+        specialArg =
+          if (args ? specialArg) then
+            # For backward compatibility only.
+            # TODO: Convert to throw after XX.XX branch-off.
+            lib.warn "mkLocalDerivation: Expect specialArg under passthru." args.specialArg
+          else
+            (_: false);
+      } // args.passthru or { };
+      # Some expressions involving specialArg
+      greeting = if finalAttrs.passthru.specialArg "hi" then "hi" else "hello";
+    };
+  removedArgNames = [
+    # Don't pass specialArg into mkDerivation.
+    "specialArg"
+  ];
+}
+```
+
+Remove the `removedArgNames` specification after deprecating `specialArg`.
+
+```nix
+lib.extendMkDerivation {
+  constructDrv = stdenv.mkDerivation;
+  extendArgs =
+    finalAttrs:
+    {
+      preferLocalBuild ? true,
+      allowSubstitute ? false,
+      ...
+    }@args:
+    # The arguments to pass
+    {
+      inherit
+        preferLocalBuild
+        allowSubstitute
+        ;
+      passthru = {
+        specialArg =
+          (lib.throwIf (args ? specialArg) "mkLocalDerivation: Expect specialArg under passthru.")
+            (_: false);
+      } // args.passthru or { };
+      # Some expressions involving specialArg
+      greeting = if finalAttrs.passthru.specialArg "hi" then "hi" else "hello";
+    };
+}
+```
+:::
diff --git a/doc/redirects.json b/doc/redirects.json
@@ -1,7 +1,19 @@
 {
+  "chap-build-helpers-finalAttrs": [
+    "index.html#chap-build-helpers-finalAttrs"
+  ],
   "chap-release-notes": [
     "release-notes.html#chap-release-notes"
   ],
+  "ex-build-helpers-extendMkDerivation-removedArgNames": [
+    "index.html#ex-build-helpers-extendMkDerivation-removedArgNames"
+  ],
+  "ex-build-helpers-extendMkDerivation-removedArgNames-migration": [
+    "index.html#ex-build-helpers-extendMkDerivation-removedArgNames-migration"
+  ],
+  "ex-build-helpers-extendMkDerivation-simple": [
+    "index.html#ex-build-helpers-extendMkDerivation-simple"
+  ],
   "neovim": [
     "index.html#neovim"
   ],
@@ -38,6 +50,12 @@
   "sec-allow-insecure": [
     "index.html#sec-allow-insecure"
   ],
+  "sec-build-helper-extendMkDerivation": [
+    "index.html#sec-build-helper-extendMkDerivation"
+  ],
+  "sec-build-helper-extendMkDerivation-removedArgNames": [
+    "index.html#sec-build-helper-extendMkDerivation-removedArgNames"
+  ],
   "sec-modify-via-packageOverrides": [
     "index.html#sec-modify-via-packageOverrides"
   ],