Skip to content

Commit

Permalink
dotnet: migrate nuget lockfiles to JSON (#362278)
Browse files Browse the repository at this point in the history
  • Loading branch information
Atemu authored Dec 18, 2024
2 parents 9c40bef + 2b4190a commit 83a5427
Show file tree
Hide file tree
Showing 304 changed files with 69,475 additions and 52,698 deletions.
65 changes: 44 additions & 21 deletions doc/languages-frameworks/dotnet.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,9 +99,9 @@ The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given
To package Dotnet applications, you can use `buildDotnetModule`. This has similar arguments to `stdenv.mkDerivation`, with the following additions:

* `projectFile` is used for specifying the dotnet project file, relative to the source root. These have `.sln` (entire solution) or `.csproj` (single project) file extensions. This can be a list of multiple projects as well. When omitted, will attempt to find and build the solution (`.sln`). If running into problems, make sure to set it to a file (or a list of files) with the `.csproj` extension - building applications as entire solutions is not fully supported by the .NET CLI.
* `nugetDeps` takes either a path to a `deps.nix` file, or a derivation. The `deps.nix` file can be generated using the script attached to `passthru.fetch-deps`. For compatibility, if the argument is a list of derivations, they will be added to `buildInputs`.
* `nugetDeps` should be a path to a JSON file, a path to a nix file (deprecated), a derivation, or a list of derivations. A `deps.json` file can be generated using the script attached to `passthru.fetch-deps`, which is the preferred method. All `nugetDeps` packages are added to `buildInputs`.
::: {.note}
For more detail about managing the `deps.nix` file, see [Generating and updating NuGet dependencies](#generating-and-updating-nuget-dependencies)
For more detail about managing the `deps.json` file, see [Generating and updating NuGet dependencies](#generating-and-updating-nuget-dependencies)
:::

* `packNupkg` is used to pack project as a `nupkg`, and installs it to `$out/share`. If set to `true`, the derivation can be used as a dependency for another dotnet project by adding it to `buildInputs`.
Expand Down Expand Up @@ -133,7 +133,7 @@ For more detail about managing the `deps.nix` file, see [Generating and updating
* `dotnetPackFlags` can be used to pass flags to `dotnet pack`. Used only if `packNupkg` is set to `true`.
* `dotnetFlags` can be used to pass flags to all of the above phases.

When packaging a new application, you need to fetch its dependencies. Create an empty `deps.nix`, set `nugetDeps = ./deps.nix`, then run `nix-build -A package.fetch-deps` to generate a script that will build the lockfile for you.
When packaging a new application, you need to fetch its dependencies. Create an empty `deps.json`, set `nugetDeps = ./deps.json`, then run `nix-build -A package.fetch-deps` to generate a script that will build the lockfile for you.

Here is an example `default.nix`, using some of the previously discussed arguments:
```nix
Expand All @@ -148,7 +148,7 @@ in buildDotnetModule rec {
src = ./.;
projectFile = "src/project.sln";
nugetDeps = ./deps.nix; # see "Generating and updating NuGet dependencies" section for details
nugetDeps = ./deps.json; # see "Generating and updating NuGet dependencies" section for details
buildInputs = [ referencedProject ]; # `referencedProject` must contain `nupkg` in the folder structure.
Expand Down Expand Up @@ -225,7 +225,7 @@ buildDotnetGlobalTool {
## Generating and updating NuGet dependencies {#generating-and-updating-nuget-dependencies}

When writing a new expression, you can use the generated `fetch-deps` script to initialise the lockfile.
After setting `nugetDeps` to the desired location of the lockfile (e.g. `./deps.nix`),
After setting `nugetDeps` to the desired location of the lockfile (e.g. `./deps.json`),
build the script with `nix-build -A package.fetch-deps` and then run the result.
(When the root attr is your package, it's simply `nix-build -A fetch-deps`.)

Expand All @@ -236,31 +236,54 @@ the upstream repository and you are inside it.
```bash
$ dotnet restore --packages out
Determining projects to restore...
Restored /home/lychee/Celeste64/Celeste64.csproj (in 1.21 sec).
Restored /home/ggg/git-credential-manager/src/shared/Git-Credential-Manager/Git-Credential-Manager.csproj (in 1.21 sec).
```

Next, use `nuget-to-nix` tool provided in nixpkgs to generate a lockfile to `deps.nix` from
Next, use `nuget-to-json` tool provided in nixpkgs to generate a lockfile to `deps.json` from
the packages inside the `out` directory.

```bash
$ nuget-to-nix out > deps.nix
$ nuget-to-json out > deps.json
```
Which `nuget-to-nix` will generate an output similar to below
```nix
{ fetchNuGet }: [
(fetchNuGet { pname = "FosterFramework"; version = "0.1.15-alpha"; hash = "sha256-lM6eYgOGjl1fx6WFD7rnRi/YAQieM0mx60h0p5dr+l8="; })
(fetchNuGet { pname = "Microsoft.AspNetCore.App.Runtime.linux-x64"; version = "8.0.1"; hash = "sha256-QbUQXjCzr8j8u/5X0af9jE++EugdoxMhT08F49MZX74="; })
(fetchNuGet { pname = "Microsoft.NET.ILLink.Tasks"; version = "8.0.1"; hash = "sha256-SopZpGaZ48/8dpUwDFDM3ix+g1rP4Yqs1PGuzRp+K7c="; })
(fetchNuGet { pname = "Microsoft.NETCore.App.Runtime.linux-x64"; version = "8.0.1"; hash = "sha256-jajBI5GqG2IIcsIMgxTHfXbMapoXrZGl/EEhShwYq7w="; })
(fetchNuGet { pname = "SharpGLTF.Core"; version = "1.0.0-alpha0031"; hash = "sha256-Bs4baD5wNIH6wAbGK4Xaem0i3luQkOQs37izBWdFx1I="; })
(fetchNuGet { pname = "SharpGLTF.Runtime"; version = "1.0.0-alpha0031"; hash = "sha256-TwJO6b8ubmwBQh6NyHha8+JT5zHDJ4dROBbsEbUaa1M="; })
(fetchNuGet { pname = "Sledge.Formats"; version = "1.2.2"; hash = "sha256-0Ddhuwpu3wwIzA4NuPaEVdMkx6tUukh8uKD6nKoxFPg="; })
(fetchNuGet { pname = "Sledge.Formats.Map"; version = "1.1.5"; hash = "sha256-hkYJ2iWIz7vhPWlDOw2fvTenlh+4/D/37Z71tCEwnK8="; })
(fetchNuGet { pname = "System.Numerics.Vectors"; version = "4.5.0"; hash = "sha256-qdSTIFgf2htPS+YhLGjAGiLN8igCYJnCCo6r78+Q+c8="; })
Which `nuget-to-json` will generate an output similar to below
```json
[
{
"pname": "Avalonia",
"version": "11.1.3",
"hash": "sha256-kz+k/vkuWoL0XBvRT8SadMOmmRCFk9W/J4k/IM6oYX0="
},
{
"pname": "Avalonia.Angle.Windows.Natives",
"version": "2.1.22045.20230930",
"hash": "sha256-RxPcWUT3b/+R3Tu5E5ftpr5ppCLZrhm+OTsi0SwW3pc="
},
{
"pname": "Avalonia.BuildServices",
"version": "0.0.29",
"hash": "sha256-WPHRMNowRnYSCh88DWNBCltWsLPyOfzXGzBqLYE7tRY="
},
// ...
{
"pname": "System.Runtime.CompilerServices.Unsafe",
"version": "6.0.0",
"hash": "sha256-bEG1PnDp7uKYz/OgLOWs3RWwQSVYm+AnPwVmAmcgp2I="
},
{
"pname": "System.Security.Cryptography.ProtectedData",
"version": "4.5.0",
"hash": "sha256-Z+X1Z2lErLL7Ynt2jFszku6/IgrngO3V1bSfZTBiFIc="
},
{
"pname": "Tmds.DBus.Protocol",
"version": "0.16.0",
"hash": "sha256-vKYEaa1EszR7alHj48R8G3uYArhI+zh2ZgiBv955E98="
}
]

```

Finally, you move the `deps.nix` file to the appropriate location to be used by `nugetDeps`, then you're all set!
Finally, you move the `deps.json` file to the appropriate location to be used by `nugetDeps`, then you're all set!

If you ever need to update the dependencies of a package, you instead do

Expand Down
6 changes: 3 additions & 3 deletions maintainers/scripts/update-dotnet-lockfiles.nix
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,9 @@
This script finds all the derivations in nixpkgs that have a 'fetch-deps'
attribute, and runs all of them sequentially. This is useful to test changes
to 'fetch-deps', 'nuget-to-nix', or other changes to the dotnet build
infrastructure. Regular updates should be done through the individual packages
update scripts.
to 'fetch-deps', 'nuget-to-nix', 'nuget-to-json', or other changes to the
dotnet build infrastructure. Regular updates should be done through the
individual packages update scripts.
*/
{ ... }@args:
import ./update.nix (
Expand Down
6 changes: 6 additions & 0 deletions nixos/doc/manual/release-notes/rl-2411.section.md
Original file line number Diff line number Diff line change
Expand Up @@ -914,6 +914,12 @@

- `virtualisation.incus` module gained new `incus-user.service` and `incus-user.socket` systemd units. It is now possible to add a user to `incus` group instead of `incus-admin` for increased security.

- `buildDotnetModule` now uses JSON-based instead of nix-based lockfiles.
Support for nix-based lockfiles has been deprecated and will be dropped on release 25.11.

- `buildDotnetModule.fetch-deps` now generates a JSON lockfile by default.
A nix-based lockfile will be generated if one existed before for compatibility, however it has been deprecated and will be dropped on release 25.11.

- `freecad` now supports addons and custom configuration in nix-way, which can be used by calling `freecad.customize`.

- `buildNimSbom` was added as an alternative to `buildNimPackage`. `buildNimSbom` uses [SBOMs](https://cyclonedx.org/) to generate packages whereas `buildNimPackage` uses a custom JSON lockfile format.
Expand Down
2 changes: 1 addition & 1 deletion pkgs/applications/audio/openutau/default.nix
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ buildDotnetModule rec {
enableParallelBuilding = false;

projectFile = "OpenUtau.sln";
nugetDeps = ./deps.nix;
nugetDeps = ./deps.json;

executables = [ "OpenUtau" ];

Expand Down
Loading

0 comments on commit 83a5427

Please sign in to comment.