Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

python3Packages: get rid of uses of nose #326513

Closed
emilazy opened this issue Jul 12, 2024 · 70 comments · Fixed by #348699
Closed

python3Packages: get rid of uses of nose #326513

emilazy opened this issue Jul 12, 2024 · 70 comments · Fixed by #348699
Labels
3.skill: sprintable A larger issue which is split into distinct actionable tasks 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: python

Comments

@emilazy
Copy link
Member

emilazy commented Jul 12, 2024 •
edited
Loading

nose is abandoned upstream and has caused us untold headaches recently (admittedly mostly unrelated to the code itself). We’ve stopped the bleeding by getting #325968 into staging thanks to the incredible work of @jchv, but it’s still broken on master and we ought to do some tidying up so we’re not dragging this around forever and perhaps even to unblock some stuff on master while the staging cycle progresses. Many packages don’t actually need nose any more, despite us including it in our dependency lists. Active upstreams should have already moved to pytest or something else. Some packages will have patches available; if we’re really feeling generous we could send patches to upstream ourselves. Others may be completely unmaintained packages that we could think about dropping.

Please make sure to migrate packages to pyproject = true; when working on them. Additionally, if the upstream is unmaintained, there are no significant reverse dependencies in‐tree or on Wheelodex, and it requires patching or there is no particular reason to imagine we’d want to keep it, consider just dropping the package. For the same reasons we want to get rid of nose, we should take this as an opportunity to reduce future maintenance burden and drop broken, unused, or long‐obsolete packages.

@dotlambda has already done some great work on this – see #325847, #325872, #325885, and #326164 – but I thought it’d be good to have a master list of all affected packages so we can tackle it collectively. If you open a PR for one of these, please edit this issue to link it beside the relevant package(s).

Already done

This was generated from staging commit 83881cb with the following awful fish one‐liner: for pkg in (rg -l '\bnose3?\b' | rg -v '/(nose[23]?|cufflinks|enocean|jaconv|annoy|hdmedians|flaky|pycron|snapshottest|ssh-mitm|colour|influxdb)/|(python|cran)-(packages|aliases)\.nix' | sed 's|.*/\([^/]*\)/\([^/]*\)$|\1|'); rg "/$pkg"'[ \n{]' pkgs/top-level/python-packages.nix | awk '{print "python3Packages." $1}'; test $pipestatus[1] = 0 || rg "/$pkg"'[ \n{]' pkgs/top-level/all-packages.nix | awk '{print $1}'; test $pipestatus[1] = 0 || echo "$pkg"; end | sort -u | sed 's/^/* [ ] `/; s/$/`/'; feel free to edit to update the list.
@emilazy emilazy added 0.kind: bug Something is broken 6.topic: python 3.skill: sprintable A larger issue which is split into distinct actionable tasks and removed 0.kind: bug Something is broken labels Jul 12, 2024
@emilazy emilazy mentioned this issue Jul 12, 2024
Closed
@AndrewKvalheim AndrewKvalheim mentioned this issue Jul 12, 2024
Merged
13 tasks
@lavafroth
Copy link
Contributor

python3Packages.biopandas has dropped pynose in v0.5.1.dev1 (not yet released). We should be able to check that off once the next version releases.

@lavafroth lavafroth mentioned this issue Jul 13, 2024
Merged
13 tasks
AndrewKvalheim added a commit to AndrewKvalheim/nixpkgs that referenced this issue Jul 13, 2024
@AndrewKvalheim
python3Packages.expiringdict: remove nose
fb0f321 
Addresses NixOS#326513 by applying mailgun/expiringdict#49
lavafroth added a commit to lavafroth/nixpkgs that referenced this issue Jul 14, 2024
@lavafroth
python3Packages.gocardless-pro: remove nose dependency
c757e22 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@cyrinux
Copy link

cyrinux commented Jul 14, 2024

Hi, "paperwork" package also depend on nose and raise error actually.

@pyrox0
Copy link
Member

pyrox0 commented Jul 14, 2024 •
edited
Loading

Seems like paperwork depends on pypillowfight, which has a build error due to nose not being supported. I'm investigating now!

Edit: it also depends on fabulous, which has some more failing tests.

@pyrox0 pyrox0 mentioned this issue Jul 14, 2024
Merged
13 tasks
@pyrox0 pyrox0 added the 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems label Jul 14, 2024
@pyrox0 pyrox0 mentioned this issue Jul 14, 2024
Merged
13 tasks
@pyrox0
Copy link
Member

pyrox0 commented Jul 15, 2024

ledger-autosync can stay broken for now, per egh/ledger-autosync#143, upstream is working on getting a new release out, which should include all fixes for distutils, nose -> pytest, etc. I could also just update to the latest commit in the meantime if that's what's wanted.

This was referenced Jul 15, 2024
Merged
Merged
@lavafroth lavafroth mentioned this issue Jul 15, 2024
Merged
13 tasks
lavafroth added a commit to lavafroth/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth
python3Packages.pastescript: remove nose dependency
e426922 
Addresses NixOS#326513.

Dependency nose was removed in version 3.3.0 (2023-01-03).
Replaced `format` parameter with `pyproject = true` and `build-system`
set to `setuptools`
d-xo pushed a commit to d-xo/nixpkgs that referenced this issue Jul 15, 2024
@AndrewKvalheim @d-xo
python3Packages.expiringdict: remove nose
f94b278 
Addresses NixOS#326513 by applying mailgun/expiringdict#49
d-xo pushed a commit to d-xo/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth @d-xo
python3Packages.gocardless-pro: remove nose dependency
18bf745 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@mindstorms6 mindstorms6 mentioned this issue Jul 15, 2024
Closed
13 tasks
shahinism pushed a commit to shahinism/nixpkgs that referenced this issue Jul 15, 2024
@AndrewKvalheim @shahinism
python3Packages.expiringdict: remove nose
c0dca70 
Addresses NixOS#326513 by applying mailgun/expiringdict#49
shahinism pushed a commit to shahinism/nixpkgs that referenced this issue Jul 15, 2024
@lavafroth @shahinism
python3Packages.gocardless-pro: remove nose dependency
b65c049 
Addresses NixOS#326513
Dependency `nose` was removed in favor of pytest in version 1.46.2.
@pyrox0
Copy link
Member

pyrox0 commented Jul 15, 2024

sasview, while listed here, does not depend on nose.

@lavafroth
Copy link
Contributor

@pyrox0 sasview has an xhtml2pdf.nix besides the default.nix which depends on nose.

@lavafroth lavafroth mentioned this issue Jul 16, 2024
Merged
13 tasks
@Sigmanificient
Copy link
Member

@emilazy should we drop more-itertools?

This was referenced Aug 21, 2024
Merged
Merged
Merged
@natsukium natsukium mentioned this issue Aug 21, 2024
Open
13 tasks
@SuperSandro2000
Copy link
Member

SuperSandro2000 commented Aug 22, 2024 •
edited
Loading

more-itertools

We can't, it is to widely used. Or do you mean the python2Package? Not sure what is even using that at this point.

@Sigmanificient
Copy link
Member

more-itertools

We can't, it is to widely used. Or do you mean the python2Package? Not sure what is even using that at this point.

yup the python2Package

@SuperSandro2000
Copy link
Member

Try removing it and see what breaks. I would be in support to drop eg https://github.com/NixOS/nixpkgs/blob/21576aba43e604a95cc45c25248f39f3989ebe36/pkgs/servers/neard/default.nix

but we need to keep gimp intact

nixpkgs/pkgs/applications/graphics/gimp/default.nix

Line 137 in 21576ab

python2.pkgs.pygtk

I didn't find anything else uses python2 packages on the fast but I didn't search for all variants.

@emilazy
Copy link
Member Author

emilazy commented Aug 22, 2024

Ignore Python 2; it is even more obsolete than nose. GIMP will be solved soon and I have plans for the rest of Python 2.

@emilazy
Copy link
Member Author

emilazy commented Sep 22, 2024

Sorry for my long absence from working on this. I’ve cleaned up the list in the first comment to be easier to follow now that most users have been dealt with, and left a few reviews. I hope we can get this done before 24.11!

@pyrox0 pyrox0 mentioned this issue Sep 22, 2024
Merged
13 tasks
@emilazy
Copy link
Member Author

emilazy commented Sep 22, 2024

PRs of my own in flight:

I’m looking at python3Packages.mhcflurry and python3Packages.uncompyle6, which are now almost the only remaining packages without an open PR.

@emilazy
Copy link
Member Author

emilazy commented Sep 22, 2024

Turns out I was wrong about Python 2.7 packages not mattering… #343822

@emilazy emilazy mentioned this issue Sep 26, 2024
Merged
13 tasks
@emilazy
Copy link
Member Author

emilazy commented Oct 11, 2024

@Sigmanificient Hi there, just checking if you plan to address the review comments on your pending PRs? No worries if not; I just want to make sure if you’d rather work more on them or would be happier for someone else to take over.

This was referenced Oct 14, 2024
Merged
Merged
Merged
emilazy added a commit to emilazy/nixpkgs that referenced this issue Oct 15, 2024
@emilazy
python3{11,12}Packages.nose: drop
ee3864b 
🎉

This package has been deprecated and unmaintained upstream for almost a
decade, has required extensive patching to keep working on new Python
versions, will inevitably break again with Python 3.13 dropping 2to3,
is lacking a maintainer in Nixpkgs, is now unused in the tree, and
has caused us all far too many headaches lately. Let’s put an end
to this!

Shout‐outs to mweinelt and jchv for dealing with this situation
early on, pyrox0, Sigmagnificent, and dotlambda for tackling a bunch
of packages, and natsukium for help with reviews. I never thought this
would get finished so quickly. We’ve collectively handled almost
1½ packages per day in the three months since I first opened the
tracking issue, and sometimes helped move the entire ecosystem forward.

Closes: NixOS#326513
emilazy added a commit to emilazy/nixpkgs that referenced this issue Oct 15, 2024
@emilazy
python3{11,12}Packages.nose: drop
d37413f 
🎉

This package has been deprecated and unmaintained upstream for almost a
decade, has required extensive patching to keep working on new Python
versions, will inevitably break again with Python 3.13 dropping 2to3,
is lacking a maintainer in Nixpkgs, is now unused in the tree, and
has caused us all far too many headaches lately. Let’s put an end
to this!

Shout‐outs to mweinelt and jchv for dealing with this situation
early on, pyrox0, Sigmanificient, and dotlambda for tackling a bunch
of packages, and natsukium for help with reviews. I never thought this
would get finished so quickly. We’ve collectively handled almost
1½ packages per day in the three months since I first opened the
tracking issue, and sometimes helped move the entire ecosystem forward.

Closes: NixOS#326513
@emilazy emilazy mentioned this issue Oct 15, 2024
Merged
13 tasks
@emilazy
Copy link
Member Author

emilazy commented Oct 15, 2024

We’ve finally reached the end of the road!

Thanks again to everyone who helped out with this immense effort.

@Sigmanificient
Copy link
Member

Sigmanificient commented Oct 15, 2024 •
edited
Loading

@Sigmanificient Hi there, just checking if you plan to address the review comments on your pending PRs? No worries if not; I just want to make sure if you’d rather work more on them or would be happier for someone else to take over.

Hey Emilazy, sadly i have almost no spare time currently :<
I've seen that you done them arleady though and congrats o everyone for finishing this tracking issue!

@emilazy
Copy link
Member Author

emilazy commented Oct 15, 2024

No worries! You played a huge part in this and I’m happy we’ve finally reached the finish line.

vinylen pushed a commit to vinylen/nixpkgs that referenced this issue Oct 17, 2024
@emilazy @vinylen
python3{11,12}Packages.nose: drop
fbf1120 
🎉

This package has been deprecated and unmaintained upstream for almost a
decade, has required extensive patching to keep working on new Python
versions, will inevitably break again with Python 3.13 dropping 2to3,
is lacking a maintainer in Nixpkgs, is now unused in the tree, and
has caused us all far too many headaches lately. Let’s put an end
to this!

Shout‐outs to mweinelt and jchv for dealing with this situation
early on, pyrox0, Sigmanificient, and dotlambda for tackling a bunch
of packages, and natsukium for help with reviews. I never thought this
would get finished so quickly. We’ve collectively handled almost
1½ packages per day in the three months since I first opened the
tracking issue, and sometimes helped move the entire ecosystem forward.

Closes: NixOS#326513
@tomodachi94 tomodachi94 mentioned this issue Dec 5, 2024
Open
@lucasew lucasew mentioned this issue Dec 17, 2024
Merged
13 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
3.skill: sprintable A larger issue which is split into distinct actionable tasks 5. scope: tracking Long-lived issue tracking long-term fixes or multiple sub-problems 6.topic: python
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

python3{11,12}Packages.nose: drop emilazy/nixpkgs
10 participants
@trofi @cyrinux @eclairevoyant @SuperSandro2000 @emilazy @pyrox0 @nixos-discourse @Sigmanificient @mmorley0395 @lavafroth