Skip to content

Commit

Permalink
Prevent double UCX initialization in test_dgx
Browse files Browse the repository at this point in the history
Double initialization of UCX context may raise exceptions and cause
test failures, prevent that by reseting the context after doing some
initial checks.
  • Loading branch information
pentschev committed Jan 5, 2024
1 parent 0f34116 commit 72b47bb
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions dask_cuda/tests/test_dgx.py
Original file line number Diff line number Diff line change
Expand Up @@ -144,6 +144,10 @@ def _test_ucx_infiniband_nvlink(
else:
skip_queue.put("ok")

# `ucp.get_active_transports()` call above initializes UCX, we must reset it
# so that Dask doesn't try to initialize it again and raise an exception.
ucp.reset()

if enable_infiniband is None and enable_nvlink is None and enable_rdmacm is None:
enable_tcp_over_ucx = None
cm_tls = ["all"]
Expand Down

0 comments on commit 72b47bb

Please sign in to comment.