Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unpin dask and distributed for 23.12 development #1264

Merged
merged 8 commits into from
Oct 27, 2023

Conversation

galipremsagar
Copy link
Contributor

This PR relaxes dask and distributed versions pinning for 23.12 development.

xref: rapidsai/cudf#14320

pentschev and others added 4 commits October 4, 2023 09:27
dask/distributed#8195 has renamed `CustomFile`
to `AnyKeyFile`, the import requires updating.
This change was introduced in
rapidsai#1247 to include the fix from
dask/distributed#8216 for the Dask-CUDA 23.10
which pinned to Distributed 2023.9.2. Starting from Distributed
2023.9.3, the fix is merged in Distributed and this isn't required
anymore.
@galipremsagar galipremsagar added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Oct 23, 2023
@galipremsagar galipremsagar self-assigned this Oct 23, 2023
@galipremsagar galipremsagar requested a review from a team as a code owner October 23, 2023 23:24
@github-actions github-actions bot added the conda conda issue label Oct 23, 2023
@pentschev pentschev requested a review from a team as a code owner October 24, 2023 13:04
@github-actions github-actions bot added the python python code needed label Oct 24, 2023
@pentschev
Copy link
Member

@galipremsagar I pushed the changes from #1256 into your branch, hope you don't mind.

@galipremsagar
Copy link
Contributor Author

@galipremsagar I pushed the changes from #1256 into your branch, hope you don't mind.

Absolutely fine by me. Thanks @pentschev !

@galipremsagar
Copy link
Contributor Author

@pentschev I think we have legit pytest failures this time, do we need to modify them?

@pentschev
Copy link
Member

I'm looking at them now, will post here shortly.

@pentschev
Copy link
Member

I'm xfailing the tests for now and filed #1265 to investigate that.

@galipremsagar
Copy link
Contributor Author

Thanks @pentschev !

@galipremsagar galipremsagar added the 5 - Ready to Merge Testing and reviews complete, ready to merge label Oct 26, 2023
@galipremsagar
Copy link
Contributor Author

/merge

@pentschev
Copy link
Member

We've got some test failures due to AWS segfaults as we're still on PyArrow 12, rerunning those.

@pentschev
Copy link
Member

This is being really hard here, pr / conda-python-tests / tests (12.0.1, ubuntu22.04, amd64, 3.10, v100, latest) failed 5 times with segfaults already:

[792bd23cf9a0:6818 :0:7196] Caught signal 11 (Segmentation fault: address not mapped to object at address 0x10)
==== backtrace (tid:   7196) ====
 0  /opt/conda/envs/test/lib/python3.10/site-packages/ucp/_libs/../../../../libucs.so.0(ucs_handle_error+0x2fd) [0x7f1dff619fed]
 1  /opt/conda/envs/test/lib/python3.10/site-packages/ucp/_libs/../../../../libucs.so.0(+0x2a1e1) [0x7f1dff61a1e1]
 2  /opt/conda/envs/test/lib/python3.10/site-packages/ucp/_libs/../../../../libucs.so.0(+0x2a3aa) [0x7f1dff61a3aa]
 3  /usr/lib/x86_64-linux-gnu/libc.so.6(+0x42520) [0x7f2080993520]
 4  /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../.././libaws-cpp-sdk-core.so(+0x824cc) [0x7f1e60d644cc]
 5  /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../../././libaws-c-io.so.1.0.0(+0x1c843) [0x7f1e60326843]
 6  /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../../././libaws-c-io.so.1.0.0(+0x167e6) [0x7f1e603207e6]
 7  /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../../././libaws-c-io.so.1.0.0(+0x16839) [0x7f1e60320839]
 8  /opt/conda/envs/test/lib/python3.10/site-packages/pyarrow/../../../././libaws-c-common.so.1(+0x26419) [0x7f1e604b3419]
 9  /usr/lib/x86_64-linux-gnu/libc.so.6(+0x94ac3) [0x7f20809e5ac3]
10  /usr/lib/x86_64-linux-gnu/libc.so.6(clone+0x44) [0x7f2080a76bf4]
=================================

@rapids-bot rapids-bot bot merged commit fbeee9c into rapidsai:branch-23.12 Oct 27, 2023
20 checks passed
pentschev added a commit to pentschev/dask-cuda that referenced this pull request Oct 27, 2023
Some tests were previously disabled in
rapidsai#1264 to prevent segfaults
that should now be resolved after upgrading to Arrow 13.
rapids-bot bot pushed a commit that referenced this pull request Oct 27, 2023
Some tests were previously disabled in #1264 to prevent segfaults that should now be resolved after upgrading to Arrow 13.

Authors:
  - Peter Andreas Entschev (https://github.com/pentschev)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)

URL: #1266
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge conda conda issue improvement Improvement / enhancement to an existing function non-breaking Non-breaking change python python code needed
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants