Skip to content

Commit

Permalink
[wip] to be squashed over the previous one [2]
Browse files Browse the repository at this point in the history
  • Loading branch information
@AndersonTorres
AndersonTorres committed Nov 6, 2024
1 parent 6aaa053 commit 5b3ba95
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions doc/hooks/versionCheckHook.section.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -69,18 +69,18 @@ Given the current limitations of Nixpkgs's tooling and CI, `versionCheckHook` fi

Below we tabulate the differences between `versionCheckHook` and `testers.testVersion`.

| Item | `versionCheckHook` | `testers.testVersion` via `passthru.tests.version` |
|------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
| Customization | Besides the attributes described above, `versionCheckHook` provides no other methods for controlling it. | `tests.version` has access to the whole feature set of Nixpkgs, like generating multiple test derivations and accessing tools outside the package inputs. |
| Execution environment | `versionCheckHook` uses `env --ignore-environment --chdir=/` to run the executables in an environment as clean as possible, but there is no way to change its behavior. | `tests.version` executes in an environment identical to that of a consumer, independent from the building environment. |
| Rebuild after modification | Modifying `versionCheckHook` triggers the package rebuild, since it is a phase executed during the the package build time. | `tests.version` does not require rebuilding the package, since it is a derivation that runs after package build time. |
| Overhead during package build time | Negligible. Although executed during the the package build time, `versionCheckHook` is lean and executes few commands. | Zero, since `tests.version` is a derivation that runs after package build time. |
| Failure detection time | `versionCheckHook` runs during the package build time, therefore it does not deal with failures happening after that. | `tests.version` detects failures after package build time, like rewritings and relocations. |
| OfBorg CI support | `versionCheckHook` will be executed by OfBorg, since it is a phase executed during the the package build time. | OfBorg supports automatic execution of `passthru.tests` for _directly affected packages_; transitive dependencies are ignored by default. |
| `nixpkgs-review` support | `versionCheckHook` will be executed by `nixpkgs-review`, since it is a phase executed during the the package build time. | `nixpkgs-review` has no support for executing `passthru.tests` at all, whether for the package or its transitive dependencies. [^1] |
| Package breakage awareness | Loud and clear as soon as `versionCheckHook` is reached during the package build time. | Requires specific commands to be noticed, e.g. `nix-build -A pkg.tests.version`. |
| Transitive dependencies awareness | Since `nixpkgs-review` executes the transitive dependencies, the ones containing `versionCheckHook` will never ever be ignored by this tool. | Since `nixpkgs-review` does not run `tests.version` and ofBorg does not execute transitive dependencies, extra measures are needed to find them. |
| Item | `versionCheckHook` | `testers.testVersion` via `passthru.tests.version` |
|-------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------|
| Customization | Besides the attributes described above, `versionCheckHook` provides no other methods for controlling it. | `tests.version` has access to the whole feature set of Nixpkgs, like multiple version test derivations and tools outside the package inputs. |
| Execution time | `versionCheckHook` is executed automatically whenever the package is executed, e.g. `nix-build -A pkg`, since it is a phase executed during the package build time. | `tests.version` is executed only when explicitly called, e.g. `nix-build -A pkg.tests.version`; it is executed after package build time. |
| Execution environment | `versionCheckHook` uses `env --ignore-environment --chdir=/` to run the executables in an environment as clean as possible, but there is no way to change its behavior. | `tests.version` executes in an environment identical to that of a consumer, independent from the package build environment. |
| Rebuild after modification | Modifying `versionCheckHook` triggers the package rebuild. | `tests.version` does not rebuild the package, since it runs after package build time. |
| Overhead during package build time | Negligible. Although executed during the the package build time, `versionCheckHook` is lean and executes few commands. | Zero, since `tests.version` is a derivation that runs after package build time. |
| Failure detection time | `versionCheckHook` runs during the package build time, therefore it does not deal with failures happening after that. | `tests.version` detects failures after package build time, like rewritings and relocations. |
| Package breakage awareness | Loud and clear as soon as `versionCheckHook` is reached during the package build time. | Requires specific commands to be noticed, e.g. `nix-build -A pkg.tests.version`. |
| OfBorg CI | `versionCheckHook` will be executed by OfBorg, since ofBorg automatically executes the packages affected by the pull request (PR). | OfBorg supports automatic execution of `passthru.tests` for _packages directly affected_ by the PR. |
| OfBorg CI, transitive dependencies | `versionCheckHook` will executed by OfBorg, since it builds transitive dependencies of a package affected by the PR. (????) | OfBorg does not support automatic execution of `passthru.tests` for transitive dependencies, requiring extra commands for this. |
| `nixpkgs-review` | `versionCheckHook` will be executed by `nixpkgs-review`. | `nixpkgs-review` has no support for executing `passthru.tests` at all. [^1] |
| `nixpkgs-review`, transitive dependencies | `versionCheckHook` will be executed by `nixpkgs-review`, since the tool reaches transitive dependencies automatically. | `nixpkgs-review` has no support for executing `passthru.tests` at all, even for transitive dependencies. [^1] |

[^1]: There is a [pull request proposal](https://github.com/Mic92/nixpkgs-review/pull/397) against [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) for adding support to `passthru.tests` execution.

As said before, given the current limitations of Nixpkgs's tooling and CI demonstrated above, `versionCheckHook` fits better in most of typical situations.

0 comments on commit 5b3ba95

Please sign in to comment.