Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[RELEASE] cugraph v24.04 #4318

Merged
merged 96 commits into from
Apr 10, 2024
Merged

[RELEASE] cugraph v24.04 #4318

merged 96 commits into from
Apr 10, 2024

Conversation

raydouglass
Copy link
Member

❄️ Code freeze for branch-24.04 and v24.04 release

What does this mean?

Only critical/hotfix level issues should be merged into branch-24.04 until release (merging of this PR).

What is the purpose of this PR?

  • Update documentation
  • Allow testing for the new release
  • Enable a means to merge branch-24.04 into main for the release

raydouglass and others added 30 commits January 18, 2024 14:59
@raydouglass
DOC v24.04 Updates [skip ci]
dd7f0a2
@GPUtester
Merge pull request #4101 from rapidsai/branch-24.02
dd8cf64 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4102 from rapidsai/branch-24.02
7d41573 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4104 from rapidsai/branch-24.02
44d3d64 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4105 from rapidsai/branch-24.02
672a037 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4110 from rapidsai/branch-24.02
77d61fc 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4113 from rapidsai/branch-24.02
49c9049 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4114 from rapidsai/branch-24.02
8fd9252 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4117 from rapidsai/branch-24.02
942d3c5 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4118 from rapidsai/branch-24.02
4e7b134 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4123 from rapidsai/branch-24.02
00f906f 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4136 from rapidsai/branch-24.02
88c3070 
Forward-merge branch-24.02 to branch-24.04
@jakirkham
Rerun ci/release/update-version.sh 24.04.00 (#4127)
26efc6d 
Looks like `ci/release/update-version.sh` may not have been run for `24.04.00`. This does that

Also noticed the following errors on macOS. Not sure if these are important. Could someone please double check?

```
sed: 1: "/^ucx_py_version:$/ {n; ...": bad flag in substitute command: '}'
sed: 1: "/^ucx_py_version:$/ {n; ...": bad flag in substitute command: '}'
sed: 1: "/^ucx_py_version:$/ {n; ...": bad flag in substitute command: '}'
```

Edit: Should add do see the right `ucx_py_version` values in these locations. So it does already appear to be correctly updated already

https://github.com/rapidsai/cugraph/blob/00f906f6e97949fd33c85143af4fdf069c791558/conda/recipes/cugraph/conda_build_config.yaml#L19-L20

https://github.com/rapidsai/cugraph/blob/00f906f6e97949fd33c85143af4fdf069c791558/conda/recipes/pylibcugraph/conda_build_config.yaml#L19-L20

https://github.com/rapidsai/cugraph/blob/00f906f6e97949fd33c85143af4fdf069c791558/conda/recipes/cugraph-service/conda_build_config.yaml#L1-L2

Authors:
   - https://github.com/jakirkham

Approvers:
   - Brad Rees (https://github.com/BradReesWork)
   - Ray Douglass (https://github.com/raydouglass)
@GPUtester
Merge pull request #4137 from rapidsai/branch-24.02
b4f7a48 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4138 from rapidsai/branch-24.02
8d3eeb8 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4139 from rapidsai/branch-24.02
29c744c 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4140 from rapidsai/branch-24.02
e2f3022 
Forward-merge branch-24.02 to branch-24.04
@GPUtester
Merge pull request #4141 from rapidsai/branch-24.02
f8f4eae 
Forward-merge branch-24.02 to branch-24.04
@vyasr
Fixes for pandas 2, latest cudf, and wheel building (#4144)
bf5aa60 
This PR contains a number of different fixes currently required to get cugraph tests passing:
- There are two main changes for pandas 2 compatibility:
    - [pandas renamed `DataFrame.applymap` to `DataFrame.map`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.map.html) so creating the renumbering map with a column `map` caused problems for attribute-based column access `renumber_map.map`. Those columns are now renamed to `renumber_map`.
    - Empty columns now default to str rather than float, so tests that assumed we could access the values as cupy arrays failed because cudf's string columns cannot be converted to cupy arrays. These columns are now always cast to float in the tests before the cupy conversion.
- cugraph-dgl and cugraph-pyg's wheel builds were not downloading the latest cugraph/pylibcugraph wheels to run tests. As a result, the above pandas 2 fixes didn't take when running the dgl and pyg tests. I updated the wheel building scripts to account for this discrepancy.
- rapidsai/cudf#14202 made a breaking change to how characters are encoded in strings columns in cudf, which broke cugraph_etl. This PR fixes the code that depended on the old APIs.

This code also includes a small patch to the cugraph_etl CMake so that it exports the correct package name (previously it was using cugraph).

Authors:
  - Vyas Ramasubramani (https://github.com/vyasr)

Approvers:
  - GALI PREM SAGAR (https://github.com/galipremsagar)
  - Bradley Dice (https://github.com/bdice)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Rick Ratzel (https://github.com/rlratzel)
  - Jake Awe (https://github.com/AyodeAwe)

URL: #4144
@GPUtester
Merge pull request #4153 from rapidsai/branch-24.02
434d808 
Forward-merge branch-24.02 to branch-24.04
@bdice
Ensure that ctest is called with --no-tests=error. (#4148)
f9e618b 
This PR ensures that all calls to `ctest` include the flag `--no-tests=error`. See rapidsai/build-planning#18.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4148
@jameslamb
handle more RAPIDS version formats in update-version.sh, refactor dep…
9ad7389 
…endencies.yaml (#4109)

Contributes to rapidsai/build-planning#13.

Updates `update-version.sh` to correctly handle RAPIDS dependencies like `cudf-cu12==24.2.*`.

This also pulls in some dependency refactoring originally added in #4088, which allows greater use of dependencies.yaml globs (and therefore less maintenance effort to support new CUDA versions).

Authors:
  - James Lamb (https://github.com/jameslamb)
  - Brad Rees (https://github.com/BradReesWork)
  - Ray Douglass (https://github.com/raydouglass)
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)
  - Bradley Dice (https://github.com/bdice)
  - Brad Rees (https://github.com/BradReesWork)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4109
@GPUtester
Merge pull request #4159 from rapidsai/branch-24.02
0e753b8 
Forward-merge branch-24.02 to branch-24.04
@seunghwak
Address primitives performance regression with the added edge masking…
38a8cdb 
… support (when edge masking is not in use) (#4119)

This addresses performance regression due to the added edge masking support when edge masking is disabled.

We addressed the issue for `per_v_transform_reduce_incoming|outgoing_e` in #4085

Using a similar approach, we address the issue for `transform_e`, `transform_reduce_e`, and `detail::nbr_intersection` in this PR.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Naim (https://github.com/naimnv)

URL: #4119
@bdice
Add cuda-nvtx-dev, add missing CUDA library dependencies. (#4162)
f63310c 
This PR fixes issues with devcontainer builds where `cuda-nvtx-dev` was missing when building `libcugraph_etl`.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)

URL: #4162
@GPUtester
Merge pull request #4164 from rapidsai/branch-24.02
9a6ad49 
Forward-merge branch-24.02 to branch-24.04
@nv-rliu
Fix FutureWarnings in Graph Creation (#4167)
fe69e7f 
Addresses #4163 

This PR uses `iloc` when referencing the index in a Series object, which removes the `FutureWarning` that was seen at graph creation.

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4167
@naimnv
Add a new notebook for SNMG benchmark runs (#4091)
b93e114 
This PR adds new notebook for SNMG benchmark runs. It
- [x]  simplifies bookkeeping and data presentation. 
- [x]  update each cugraph function wrappers to handle both SG and MG calls depending on a flag. The object is to have a common set of function wrappers for SG, SNMG and MNMG and put them in a common module to avoid duplication.

Authors:
  - Naim (https://github.com/naimnv)
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Don Acosta (https://github.com/acostadon)
  - Brad Rees (https://github.com/BradReesWork)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4091
@naimnv
Use input rng_state rather than creating a local one (#4168)
3b927f6 
Use input rng_state rather than creating a one local to function

Authors:
  - Naim (https://github.com/naimnv)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4168
@jameslamb
Support CUDA 12.2 (#4088)
8f31e43 
* switches to CUDA 12.2.2 for building conda packages and wheels
* adds new tests running against CUDA 12.2.2

### Notes for Reviewers

This is part of ongoing work to build and test packages against CUDA 12.2.2 across all of RAPIDS.

For more details see:

* rapidsai/build-planning#7
* rapidsai/shared-workflows#166

Planning a second round of PRs to revert these references back to a proper `branch-24.{nn}` release branch of `shared-workflows` once rapidsai/shared-workflows#166 is merged.

*(created with `rapids-reviser`)*

Authors:
  - James Lamb (https://github.com/jameslamb)
  - Bradley Dice (https://github.com/bdice)
  - Vyas Ramasubramani (https://github.com/vyasr)
  - https://github.com/jakirkham

Approvers:
  - Jake Awe (https://github.com/AyodeAwe)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4088
seunghwak and others added 13 commits March 14, 2024 00:29
@seunghwak
Fix kv_store_t build error when cuCollection is used as a backend and…
a410b3f 
… the value type is thrust::tuple (#4238)

Fix build errors.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4238
@nv-rliu
Refactor MG Centrality Tests (#4197)
fa096f0 
Addresses #4187 

This PR makes improvements to old testing conventions used in `cugraph.centrality` MG tests

Instead of using nested fixtures. eg. `input_expected_output -> input_combo -> fixture_params` which can be confusing, the `@pytest.mark.parametrize` marker is used to iterate through combinations of parameters used for testing. The fixtures are also replaced by functions used to create SG and MG graphs.

Authors:
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Don Acosta (https://github.com/acostadon)
  - Rick Ratzel (https://github.com/rlratzel)
  - Joseph Nke (https://github.com/jnke2016)

URL: #4197
@rlratzel
Removes (unintentional?) runtime dependency on packaging from nx-cu…
6d45e7b 
…graph (#4240)

A [recent PR](#4217) was merged that added `packaging` as a runtime dependency to nx-cugraph.  This PR removes that dependency and manually extracts the `major` and `minor` version numbers instead.

Authors:
  - Rick Ratzel (https://github.com/rlratzel)
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Erik Welch (https://github.com/eriknw)

URL: #4240
@naimnv
cugraph examples (#4128)
1be5e27 
This PR adds example on how to use libcugraph to run different algorithms. 

Closes #2676

Authors:
  - Naim (https://github.com/naimnv)
  - Ralph Liu (https://github.com/nv-rliu)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Joseph Nke (https://github.com/jnke2016)
  - Don Acosta (https://github.com/acostadon)
  - Seunghwa Kang (https://github.com/seunghwak)

URL: #4128
@eriknw
nx-cugraph: add is_negatively_weighted (#4242)
7aeaec9 
Authors:
  - Erik Welch (https://github.com/eriknw)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4242
@mroeschke
Enable pytest failures on warnings from cudf (#4223)
a467da9 
In 24.04, cudf issues a FutureWarning when using positional indexing with `__getitem__`. The change here is to use `iloc` instead.

Additionally the test suite will now fail when a warning comes from cudf as apart of rapidsai/build-planning#26

Authors:
  - Matthew Roeschke (https://github.com/mroeschke)
  - Ralph Liu (https://github.com/nv-rliu)
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Don Acosta (https://github.com/acostadon)

URL: #4223
@naimnv
update example build script with targets and options/help (#4254)
7904ff0 
update example build script with targets and options/help

Authors:
  - Naim (https://github.com/naimnv)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4254
@bdice
Remove outdated conda_build.sh. (#4257)
47cd8c7 
This PR removes the `conda_build.sh` script from the repository root. This script appears to be outdated and unusable with the current cugraph recipes.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: #4257
@seunghwak
Refactor test utility functions and update SG & MG core_number tests …
d6fa29e 
…to test with edge masking (#4253)

We previously had `test_utilities.hpp` having all different sorts of function declarations and `property_generator.cuh` (which is necessary to create an edge mask) was only for primitives testing.

This PR divides `test_utilities.hpp` to multiple files (`check_utilities.hpp`, `conversion_utilities.hpp`, `csv_file_utilities.hpp`, `matrix_market_file_utilities.hpp`, and `misc_utilties.hpp`) renamed/moved `tests/prims/property_generator.cuh` to `tests/utilities/property_generator_utilities.hpp` (now this file can be included from .cpp files).

Updated SG & MG core_number tests to use property generator to create an edge mask.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Naim (https://github.com/naimnv)
  - Joseph Nke (https://github.com/jnke2016)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: #4253
@tingyu66
Always scatter in FP32 in TPConv (#4260)
4becbe8 
Always scatter in FP32 in TPConv due to poor performance of torch's native scatter_add in bfloat16.

Authors:
  - Tingyu Wang (https://github.com/tingyu66)

Approvers:
  - Alex Barghi (https://github.com/alexbarghi-nv)

URL: #4260
@KyleFromNVIDIA
Update pre-commit-hooks to v0.0.3 (#4264)
8706115 
This fixes an issue with how the `verify-copyright` hook handles multiple merge bases.

Authors:
  - Kyle Edwards (https://github.com/KyleFromNVIDIA)

Approvers:
  - Don Acosta (https://github.com/acostadon)

URL: #4264
@jnke2016
Ktruss implementation (#4059)
f753e51 
Implements SG and MG ktruss using graph primitives and drop `cuHornet`.

Closes #3447
Closes #3448 
Closes #3449 
Closes #3450 
Closes #3451
Closes #3452 
Closes #3453

Authors:
  - Joseph Nke (https://github.com/jnke2016)
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Seunghwa Kang (https://github.com/seunghwak)
  - Rick Ratzel (https://github.com/rlratzel)

URL: #4059
@bdice
Use conda env create --yes instead of --force (#4286)
dad2c2e 
conda dropped support for the `--force` flag to `conda env create`. This changes that flag name to `--yes`.
See https://github.com/conda/conda/blob/main/CHANGELOG.md#2430-2024-03-12 and rapidsai/miniforge-cuda#63 for more info.

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)
  - https://github.com/jakirkham

URL: #4286
@raydouglass raydouglass requested review from a team as code owners April 4, 2024 20:23
@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Apr 4, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@review-notebook-app Review Notebook App
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@raydouglass raydouglass merged commit a920470 into main Apr 10, 2024
3 of 5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@BradReesWork BradReesWork BradReesWork approved these changes

Assignees
No one assigned
Labels
benchmarks ci CMake conda cuGraph datasets python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.