Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Try pkg.meta.position before unsafeGetAttrPos #247

Merged
merged 1 commit into from
May 8, 2024

Conversation

pbsds
Copy link
Contributor

@pbsds pbsds commented Apr 30, 2024

Some package builders set meta.position manually due to clobbering the origin of src.

Fixes nix-update -u mpvScripts.*, discovered while reviewing NixOS/nixpkgs#308062.

@pbsds pbsds force-pushed the use-meta-position branch 2 times, most recently from e2d9ad0 to 1aede84 Compare May 4, 2024 15:11
else if (builtins.unsafeGetAttrPos "src" pkg) != null then
sanitizePosition (builtins.unsafeGetAttrPos "src" pkg)
else
sanitizePosition (positionFromMeta pkg);
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we have meta.position, should it not be preferred over src?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i was trying to change as little of existing behaviour as possible, by if you're on board with this change then why not.

@pbsds pbsds force-pushed the use-meta-position branch from 1aede84 to 2261701 Compare May 6, 2024 21:44
@pbsds pbsds changed the title Fallback to pkg.meta.position if unsafeGetAttrPos returns null Try pkg.meta.position before (unsafeGetAttrPos "src" pkg) May 6, 2024
@@ -142,13 +142,22 @@ def eval_expression(
return f"""
let
{indent(dedent(let_bindings), " ")}
positionFromMeta = pkg: let
parts = builtins.match "(.*):([0-9]+)" pkg.meta.position;
in if pkg ? meta.position then {{
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need this if here? It's already checked on line 158. Also would make the code slightly simpler.

@pbsds pbsds force-pushed the use-meta-position branch from 2261701 to 8b8f311 Compare May 7, 2024 23:26
raw_version_position = sanitizePosition (builtins.unsafeGetAttrPos "version" pkg);

position = if pkg ? isRubyGem then
raw_version_position
else if pkg ? isPhpExtension then
raw_version_position
else
else if pkg ? meta.position then
sanitizePosition (positionFromMeta pkg)
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can even overrule the quirks for ruby gems and php here, I would say.

@pbsds pbsds force-pushed the use-meta-position branch from 8b8f311 to 21f8122 Compare May 8, 2024 21:48
@pbsds pbsds changed the title Try pkg.meta.position before (unsafeGetAttrPos "src" pkg) Try pkg.meta.position before unsafeGetAttrPos May 8, 2024
Fixes `nixpkgs.mpvScripts.*`, discovered while reviewing NixOS/nixpkgs#308062
@pbsds pbsds force-pushed the use-meta-position branch from 21f8122 to 97e6d33 Compare May 8, 2024 21:49
@Mic92
Copy link
Owner

Mic92 commented May 8, 2024

@mergify queue

Copy link
Contributor

mergify bot commented May 8, 2024

queue

✅ The pull request has been merged automatically

The pull request has been merged automatically at 63b5a4e

@mergify mergify bot merged commit 63b5a4e into Mic92:master May 8, 2024
4 checks passed
Copy link
Contributor

@doronbehar doronbehar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR broke my updating workflow for gnuradioMinimal :( I don't set manually meta.position, but I did set src in the file where the hash and version are actually written... Also, setting the meta.position manually doesn't work for me from some reason.

@Mic92
Copy link
Owner

Mic92 commented Jul 24, 2024

what is meta.position set to in this case?

@doronbehar
Copy link
Contributor

To be more specific, here's my (Nixpkgs) patching attempt of trying to set the meta.position manually:

diff --git i/pkgs/applications/radio/gnuradio/wrapper.nix w/pkgs/applications/radio/gnuradio/wrapper.nix
index f468aadb27b5..107efe90bc80 100644
--- i/pkgs/applications/radio/gnuradio/wrapper.nix
+++ w/pkgs/applications/radio/gnuradio/wrapper.nix
@@ -163,6 +163,14 @@ let
     ;
     pkgs = packages;
   };
+  meta = unwrapped.meta // {
+    position = ./. + (
+      if unwrapped.versionAttr.major == "3.10" then
+        "default"
+      else
+        unwrapped.versionAttr.major
+      ) + ".nix:0";
+  };
   self = if doWrap then
     stdenv.mkDerivation {
       inherit pname version passthru;
@@ -197,11 +205,12 @@ let
           wrapProgram "$i" ${makeWrapperArgs}
         done
       '';
-      inherit (unwrapped) meta;
+      inherit meta;
     }
   else
     unwrapped.overrideAttrs(_: {
       inherit passthru;
+      inherit meta;
     })
   ;
 in self

With and without this patch, this command prints the same output:

nix eval --json -L --impure --expr '
  let
    pkgs = (builtins.getFlake "'$PWD'").legacyPackages.${builtins.currentSystem}; 
  in
    pkgs.gnuradio.meta.position
  ' | jq --raw-output '.'

Output:

/nix/store/gzkvmymnwsd2r3ypzyxfpn7w4vsmjxqq-source/pkgs/applications/radio/gnuradio/shared.nix:127

Instead of the expected:

/nix/store/gzkvmymnwsd2r3ypzyxfpn7w4vsmjxqq-source/pkgs/applications/radio/gnuradio/default.nix:0

@Mic92
Copy link
Owner

Mic92 commented Jul 24, 2024

Does lib.traceVal print your meta.position/meta at all during eval?

@doronbehar
Copy link
Contributor

doronbehar commented Jul 24, 2024

If you meant this:

nix eval --json -L --impure --expr '
  let
    pkgs = (builtins.getFlake "'$PWD'").legacyPackages.${builtins.currentSystem}; 
  in
    pkgs.lib.traceVal pkgs.gnuradio.meta.position
  ' | jq --raw-output '.'

Then the output is:

trace: /nix/store/ram9z8s3vawgf6hkf5wm5q2jlwlyi401-source/pkgs/applications/radio/gnuradio/shared.nix:127
/nix/store/ram9z8s3vawgf6hkf5wm5q2jlwlyi401-source/pkgs/applications/radio/gnuradio/shared.nix:127

@doronbehar
Copy link
Contributor

Even for the simplest package:

diff --git i/pkgs/by-name/he/hello/package.nix w/pkgs/by-name/he/hello/package.nix
index 7b8db4c7c3c2..c96f372399d8 100644
--- i/pkgs/by-name/he/hello/package.nix
+++ w/pkgs/by-name/he/hello/package.nix
@@ -52,6 +52,7 @@ stdenv.mkDerivation (finalAttrs: {
     license = licenses.gpl3Plus;
     maintainers = [ maintainers.eelco ];
     mainProgram = "hello";
+    position = "bla.nix";
     platforms = platforms.all;
   };
 })

This command:

nix eval --json -L --impure --expr '
  let
    pkgs = (builtins.getFlake "'$PWD'").legacyPackages.${builtins.currentSystem}; 
  in
    pkgs.lib.traceVal pkgs.hello.meta.position
  ' | jq --raw-output '.'

Prints:

trace: /nix/store/g6inivzr0ji1ii58b3qyxaxsaz8my5xh-source/pkgs/by-name/he/hello/package.nix:45
/nix/store/g6inivzr0ji1ii58b3qyxaxsaz8my5xh-source/pkgs/by-name/he/hello/package.nix:45

@pbsds
Copy link
Contributor Author

pbsds commented Jul 24, 2024

we could move the meta.position location to have the lowest priority

@doronbehar
Copy link
Contributor

we could move the meta.position location to have the lowest priority

That would work for me probably. I hope though that it won't break your setup with the mpvScripts - a setup which I don't understand how it used the meta.position attribute - I Read a bit the related Nix files, like buildLua.nix, and still didn't manage to figure this out myself - a few examples like the above with nix eval commands would be great.

raw_version_position
else if pkg ? isPhpExtension then
raw_version_position
else
else
sanitizePosition (builtins.unsafeGetAttrPos "src" pkg);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed if it is possible to set meta.position, this nix-update PR here makes a bit more sense. However, since mkDerivation always sets meta.position, the pkg.src attribute will never be retrieved here. Or perhaps it will be retrieved for other derivation builders? I doubt if there are builders that are not based on mkDerivation for which it will happen.

@azuwis
Copy link

azuwis commented Aug 10, 2024

This also break packages defined by overrideAttrs, like renode-unstable.

nix-update 0.14.0:

$ cd ~/src/nixpkgs
$ nix shell github:NixOS/nixpkgs/b60793b86201040d9dee019a05089a9150d08b5b#nix-update
$ nix-update renode-unstable --version=1.15.1+20240808git7a138330e
...
Update 1.15.1+20240801git19eb5fb22 -> 1.15.1+20240808git7a138330e in /home/azuwis/src/nixpkgs/pkgs/by-name/re/renode/package.nix
$ nix-build --expr let src = (import "/home/azuwis/src/nixpkgs" (if (builtins.hasAttr "config" (builtins.functionArgs (import "/home/azuwis/src/nixpkgs"))) then { config.checkMeta = false; overlays = []; } else { }))."renode-unstable".src; in (src.overrideAttrs or (f: src // f src)) (_: { outputHash = ""; outputHashAlgo = "sha256"; }) --extra-experimental-features flakes nix-command
Package maintainers:
  - Otavio Salvador (@otavio)
$ git -C /home/azuwis/src/nixpkgs diff -- /home/azuwis/src/nixpkgs/pkgs/by-name/re/renode/package.nix
No changes detected, skipping remaining steps

nix-update 0.13.1:

$ exit
$ nix shell github:NixOS/nixpkgs/b6bcda5de105b202e7ef53af5109d5f256b1181d#nix-update
$ nix-update renode-unstable --version=1.15.1+20240808git7a138330e
...
Update 1.15.1+20240801git19eb5fb22 -> 1.15.1+20240808git7a138330e in /home/azuwis/src/nixpkgs/pkgs/by-name/re/renode-unstable/package.nix
$ nix-build --expr let src = (import "/home/azuwis/src/nixpkgs" (if (builtins.hasAttr "config" (builtins.functionArgs (import "/home/azuwis/src/nixpkgs"))) then { config.checkMeta = false; overlays = []; } else { }))."renode-unstable".src; in (src.overrideAttrs or (f: src // f src)) (_: { outputHash = ""; outputHashAlgo = "sha256"; }) --extra-experimental-features flakes nix-command
Package maintainers:
  - Otavio Salvador (@otavio)
$ git -C /home/azuwis/src/nixpkgs diff -- /home/azuwis/src/nixpkgs/pkgs/by-name/re/renode-unstable/package.nix
$ git diff
diff --git a/pkgs/by-name/re/renode-unstable/package.nix b/pkgs/by-name/re/renode-unstable/package.nix
index 1510d352767a..746a18168685 100644
--- a/pkgs/by-name/re/renode-unstable/package.nix
+++ b/pkgs/by-name/re/renode-unstable/package.nix
@@ -5,11 +5,11 @@
 
 renode.overrideAttrs (finalAttrs: _: {
   pname = "renode-unstable";
-  version = "1.15.1+20240801git19eb5fb22";
+  version = "1.15.1+20240808git7a138330e";
 
   src = fetchurl {
     url = "https://builds.renode.io/renode-${finalAttrs.version}.linux-dotnet.tar.gz";
-    hash = "sha256-dIyMQtFXvHivlzC+Y3TrWsN81/cETKTaucZY5r/x5rU=";
+    hash = "sha256-hxGh+Pzpvw7dfRLdaqSEUCM8zLN9z2HQD8owOCu/uY4=";
   };
 
   passthru.updateScript =

Notice nix-update 0.14.0 try to update file nixpkgs/pkgs/by-name/re/renode/package.nix, which is wrong, and produces no changes.

While nix-update 0.13.0 updates file nixpkgs/pkgs/by-name/re/renode-unstable/package.nix, and gives correct result.

@pbsds
Copy link
Contributor Author

pbsds commented Aug 10, 2024

I revert back to my initial proposal in #267, please test it out and give us feedback :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants