Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add compile time check to ensure the counting_iterator type in counting_transform_iterator fits in size_type #17118

Conversation

mhaseeb123
Copy link
Member

@mhaseeb123 mhaseeb123 commented Oct 17, 2024

Description

This PR adds a compile time check to enforce that the start argument to cudf::detail::counting_transform_iterator, which is used to determine the type of counting_iterator, is of a type that fits in int32_t (aka size_type). The PR also modifies the instances of counting_transform_iterator that need to work with counting_iterators of type > int32_t to manually created counting_transform_iterators using thrust.

More context in this comment.

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@github-actions github-actions bot added the libcudf Affects libcudf (C++/CUDA) code. label Oct 17, 2024
@mhaseeb123 mhaseeb123 self-assigned this Oct 17, 2024
@mhaseeb123 mhaseeb123 added feature request New feature or request 2 - In Progress Currently a work in progress improvement Improvement / enhancement to an existing function non-breaking Non-breaking change and removed feature request New feature or request labels Oct 17, 2024
@mhaseeb123 mhaseeb123 added 3 - Ready for Review Ready for review by team and removed 2 - In Progress Currently a work in progress labels Oct 17, 2024
@mhaseeb123 mhaseeb123 marked this pull request as ready for review October 17, 2024 22:35
@mhaseeb123 mhaseeb123 requested a review from a team as a code owner October 17, 2024 22:35
Copy link
Contributor

@vuule vuule left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is a good change, even if it is a bit breaking. Reminds me of the surprising return value type in std::accumulate, which is determined using the initial value, rather than the type that the binaryOp takes or returns.
I think this is worth a PSA, to make sure the size_type conversion is not actually a desired feature.

@davidwendt
Copy link
Contributor

The intention was to use the transform iterator with thrust and since we only use the 32-bit bound thrust/cub calls, this was a good reminder/check not to use more than size_type values with this iterator. Also, the predicate was usually addressing column rows or similar features which is limited to size_type as well.
I feel this may cause more problems and give the appearance that the iterator supports more than it should in practice. It is a simple wrapper on the thrust::make_transform_iterator so if there is a real need to use more than 32-bit values, just use the thrust one instead. We could document this better as well.

@mhaseeb123
Copy link
Member Author

The intention was to use the transform iterator with thrust and since we only use the 32-bit bound thrust/cub calls, this was a good reminder/check not to use more than size_type values with this iterator. Also, the predicate was usually addressing column rows or similar features which is limited to size_type as well.

I see. Yeah it makes sense in that case.

I feel this may cause more problems and give the appearance that the iterator supports more than it should in practice. It is a simple wrapper on the thrust::make_transform_iterator so if there is a real need to use more than 32-bit values, just use the thrust one instead. We could document this better as well.

I agree but perhaps we should add some compile-time check as well in addition to documenting it?

@mhaseeb123 mhaseeb123 changed the title Templatize the start argument to counting_transform_iterator Add compile time check to ensure the start argument to counting_transform_iterator fits in size_type Oct 18, 2024
@mhaseeb123 mhaseeb123 changed the title Add compile time check to ensure the start argument to counting_transform_iterator fits in size_type Add compile time check to ensure the counting_iterator type in counting_transform_iterator fits in size_type Oct 18, 2024
cpp/include/cudf/detail/iterator.cuh Outdated Show resolved Hide resolved
cpp/include/cudf/detail/iterator.cuh Outdated Show resolved Hide resolved
@mhaseeb123 mhaseeb123 requested a review from vuule October 18, 2024 21:34
Copy link
Contributor

@kingcrimsontianyu kingcrimsontianyu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm! Left a comment about using static_assert.

@mhaseeb123 mhaseeb123 added 5 - Ready to Merge Testing and reviews complete, ready to merge and removed 3 - Ready for Review Ready for review by team labels Oct 21, 2024
@mhaseeb123
Copy link
Member Author

/merge

@rapids-bot rapids-bot bot merged commit 13de3c1 into rapidsai:branch-24.12 Oct 22, 2024
101 checks passed
@mhaseeb123 mhaseeb123 deleted the fea/templatize-counting-transform-iter-for-int64 branch October 22, 2024 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge improvement Improvement / enhancement to an existing function libcudf Affects libcudf (C++/CUDA) code. non-breaking Non-breaking change
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants