Skip to content

Commit

Permalink
Temporarily disable compression for communication protocols
Browse files Browse the repository at this point in the history
For GPU data, compression is worse rather than better because it
provokes device-to-host transfers when they are unnecessary.

This is a short-term fix for rapidsai#935, in lieu of hooking up GPU-based
compression algorithms.
  • Loading branch information
wence- committed Jul 21, 2022
1 parent 435dae8 commit 0e2b75b
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions dask_cuda/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,3 +31,7 @@
dask.dataframe.shuffle.shuffle_group
)
dask.dataframe.core._concat = unproxify_decorator(dask.dataframe.core._concat)

# Until GPU-based compression is hooked up, turn off compression
# in communication protocols (see https://github.com/rapidsai/dask-cuda/issues/935)
dask.config.config["distributed"]["comm"]["compression"] = None

0 comments on commit 0e2b75b

Please sign in to comment.