Amend non-reinstallable packages list

[adamgundry]

cabal-install maintains a list of nonReinstallablePackages, preventing certain packages being reinstalled by the user. This list needs some updates:

• ghc-internal (released with GHC 9.10) should be considered non-reinstallable, so it needs to be added to the list.
• Following the excellent work by @TeofilC in https://gitlab.haskell.org/ghc/ghc/-/merge_requests/12479, GHC 9.12 will allow reinstallation of the template-haskell package, so it can be removed from the list (cf. Mark template-haskell non-upgradable (again) #4185).
• GHC 9.12 will hopefully allow reinstallation of base; this is not yet fully implemented but is tracked in https://gitlab.haskell.org/ghc/ghc/-/issues/24903 (cc @mpickering).

The list of non-reinstallable packages does not currently depend on the compiler version. Really it should; is there an easy way to achieve this?

cabal/cabal-install/src/Distribution/Client/Dependency.hs

Lines 447 to 471 in dda541c

	-- | The set of non-reinstallable packages includes those which cannot be
	-- rebuilt using a GHC installation and Hackage-published source distribution.
	-- There are a few reasons why this might be true:
	--
	-- * the package overrides its unit ID (e.g. with ghc's @-this-unit-id@ flag),
	-- which can result in multiple indistinguishable packages (having potentially
	-- different ABIs) with the same unit ID.
	--
	-- * the package contains definitions of wired-in declarations which tie
	-- it to a particular compiler (e.g. we can't build link against
	-- @base-4.18.0.0@ using GHC 9.6.1).
	--
	-- * the package does not have a complete (that is, buildable) source distribution.
	-- For instance, some packages provided by GHC rely on files outside of the
	-- source tree generated by GHC's build system.
	nonReinstallablePackages :: [PackageName]
	nonReinstallablePackages =
	[ mkPackageName "base"
	, mkPackageName "ghc-bignum"
	, mkPackageName "ghc-prim"
	, mkPackageName "ghc"
	, mkPackageName "integer-gmp"
	, mkPackageName "integer-simple"
	, mkPackageName "template-haskell"
	]

[adamgundry]

I've opened #10093 to add ghc-internal to the list, since that is otherwise wrong for GHC 9.10. We don't need to resolve base and template-haskell just yet.

Ideally we would keep template-haskell in the list for existing GHC versions but remove it for future versions. Perhaps GHC should expose some interface whereby cabal-install can query the list of non-reinstallable packages?

[geekosaur]

I've been thinking it would be nice of the libdir contained a file of bootlibs and dependencies, which would allow us to both get the non-reinstallable libs and treat the others as non-reinstallable if ghc is in the closure.

[("base", Nothing), -- non-reinstallable in this version
 ("ghc", Just […]) -- list of packages that are non-reinstallable if "ghc" is used
]

(quick example only, I don't know enough about the details)

[geekosaur]

Also, do we need to worry about the likes of ghc-internal?
-- ETA: wrong one, I saw several new internal libs in the list. ghc-boot, ghc-boot-th, ghc-experimental.

[phadej]

Perhaps GHC should expose some interface whereby cabal-install can query the list of non-reinstallable packages?

The issue is that packages which cannot be actually built are uploaded to Hackage. AFAICT, the only reason is to provide haddocks in the common place with everything else. But I'd argue it's primarily a Hackage issue. If say base-4.20.0.0 or ghc-internal-9.1001.1 are not reinstallable packages, they shouldn't be in Hackage index (= the input to solver) to begin with.

[mpickering]

@geekosaur Package distributed with ghc are reinstallable unless the build plan contains ghc. The reason that already installed packages are preferred is #9669 which is unrelated to this ticket as far as I can tell.

[geekosaur]

"The build plan contains ghc" is exactly the case I want to capture. I think I missed a case, though: Just [] captures things like text which are reinstallable.

[TeofilC]

I think we only need the first half of your tuple @geekosaur, the second part is just the reverse dependencies of the package from the list, and I think that should be covered already.

I've made a ticket to track the GHC side of this: https://gitlab.haskell.org/ghc/ghc/-/issues/25282

[geekosaur]

The second would matter if the bootlibs started to include a package with a partial or disjoint set of dependencies; I was thinking in terms of forward compatibility.

[TeofilC]

The second would matter if the bootlibs started to include a package with a partial or disjoint set of dependencies; I was thinking in terms of forward compatibility.

I see that makes sense. I feel like maybe we could just go with the simpler thing for now. And if we need this in the future we could modify the interface. I don't imagine we will be needing this extra feature any time soon, since you normally want all the dependencies as well, otherwise you'd currently get broken packages.

[TeofilC]

To update on this. I made an initial MR here: https://gitlab.haskell.org/ghc/ghc/-/merge_requests/13297

@mpickering made some great points and it's likely that this interface will need more thought as GHC shouldn't expose information about packages, as that's outside its area of concern.

I'm taking a break from this right now (someone else is free to pick it up), but I'm hoping to get back to it before 9.14

[geekosaur]

Insifar as it's ghc that is foisting this set of packages and their special requirements on the community, I must disagree.