Conda-forge packaging dependencies

[ericpre]

I have opened a pull request to the conda-forge feedstock of dask-cuda to avoid installing unnecessary dependency (pyarrow, bokeh) by removing dask and keeping dask-core: conda-forge/dask-cuda-feedstock#26.
As pyarrow and bokeh are not defined in

dask-cuda/pyproject.toml

Lines 18 to 26 in e728eb6

	dependencies = [
	"dask >=2023.5.1",
	"distributed >=2023.5.1",
	"pynvml >=11.0.0,<11.5",
	"numpy >=1.21",
	"numba >=0.57",
	"pandas >=1.3,<1.6.0dev0",
	"zict >=2.0.0",
	]

they will not be installed when installing from pypi but they will when installed from conda-forge.

@jakirkham suggested to open an issue to discuss here. As I understand, this is a conda-forge packaging only concern but it doesn't hurt opening an issue to discuss it!

What do you think?

[pentschev]

I agree with this change. None of these dependencies are really necessary and making them optional makes the most sense.

[jakirkham]

Currently we are using pyproject.toml as the source of truth for both wheels & conda packages, which causes this confusion on the dask dependency

Think we should move to dependencies.yaml and borrow cuDF's approach to encoding the Dask dependency

If we make that change, then it will be straightforward to have only dask-core in the conda case and dask in the wheel case

[ericpre]

What is the status of this issue?

[jakirkham]

These are the currently list dependencies

dask-cuda/pyproject.toml

Lines 18 to 26 in 897f0b6

	dependencies = [
	"click >=8.1",
	"numba>=0.57",
	"numpy>=1.21",
	"pandas>=1.3",
	"pynvml>=11.0.0,<11.5",
	"rapids-dask-dependency==24.4.*",
	"zict>=2.0.0",
	] # This list was generated by `rapids-dependency-file-generator`. To make changes, edit dependencies.yaml and run `rapids-dependency-file-generator`.

Updating the recipe in conda-forge to a more recent version would allow for cleanup to occur. There is some work needed in conda-forge to update dask-cuda ( conda-forge/dask-cuda-feedstock#29 (comment) )