Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Migrate ORC reader to pylibcudf #16042

Merged
merged 11 commits into from
Sep 25, 2024
Merged

Migrate ORC reader to pylibcudf #16042

merged 11 commits into from
Sep 25, 2024

Conversation

lithomas1
Copy link
Contributor

Description

xref #15162

Checklist

  • I am familiar with the Contributing Guidelines.
  • New or existing tests cover these changes.
  • The documentation is up to date with these changes.

@lithomas1 lithomas1 added feature request New feature or request non-breaking Non-breaking change labels Jun 15, 2024
@github-actions github-actions bot added Python Affects Python cuDF API. CMake CMake build issue pylibcudf Issues specific to the pylibcudf package labels Jun 15, 2024
@lithomas1 lithomas1 mentioned this pull request Jul 8, 2024
Closed
@lithomas1 lithomas1 changed the base branch from branch-24.08 to branch-24.10 July 25, 2024 20:10
@vyasr
Copy link
Contributor

vyasr commented Aug 20, 2024

I'll rebase this once a couple more of the open Arrow changes go in since some of them touch the orc files.

@lithomas1
Copy link
Contributor Author

Thanks, this was mostly ready to go last time.

Just needed more tests.

@vyasr vyasr force-pushed the pylibcudf-io-orc branch from 97e740c to ee72ff5 Compare August 28, 2024 23:56
@copy-pr-bot copy-pr-bot
Copy link

copy-pr-bot bot commented Aug 28, 2024

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot removed the pylibcudf Issues specific to the pylibcudf package label Aug 28, 2024
@vyasr vyasr force-pushed the pylibcudf-io-orc branch from 448dcbd to 748f11c Compare August 29, 2024 00:02
@vyasr
Copy link
Contributor

vyasr commented Aug 29, 2024

/ok to test

@vyasr vyasr force-pushed the pylibcudf-io-orc branch from 748f11c to 79ca12b Compare August 29, 2024 00:04
@vyasr
Copy link
Contributor

vyasr commented Aug 29, 2024

/ok to test

@vyasr vyasr marked this pull request as ready for review August 29, 2024 00:08
@vyasr vyasr requested a review from a team as a code owner August 29, 2024 00:08
@vyasr
Copy link
Contributor

vyasr commented Aug 29, 2024

/ok to test

@vyasr
Copy link
Contributor

vyasr commented Aug 29, 2024

/ok to test

bdice
bdice reviewed Aug 29, 2024
View reviewed changes
Copy link
Contributor

@bdice bdice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few suggestions. Generally this seems fine. Code movement into pylibcudf is good!

python/cudf/cudf/_lib/orc.pyx Outdated Show resolved Hide resolved
python/cudf/cudf/_lib/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/tests/common/utils.py Show resolved Hide resolved
wence-
wence- requested changes Sep 3, 2024
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few small niggles

python/pylibcudf/pylibcudf/io/orc.pyx Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/io/orc.pyx Outdated Show resolved Hide resolved
python/pylibcudf/pylibcudf/tests/io/test_csv.py Outdated Show resolved Hide resolved
@vyasr
Copy link
Contributor

vyasr commented Sep 3, 2024

/ok to test

@vyasr vyasr requested review from bdice and wence- September 3, 2024 22:25
wence-
wence- requested changes Sep 20, 2024
View reviewed changes
Copy link
Contributor

@wence- wence- left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is basically ready apart from the niggles around default types and docstring suggestions from @bdice

@github-actions github-actions bot added the pylibcudf Issues specific to the pylibcudf package label Sep 21, 2024
@vyasr
Copy link
Contributor

vyasr commented Sep 21, 2024

/ok to test

@vyasr vyasr requested a review from wence- September 21, 2024 05:19
@galipremsagar
Copy link
Contributor

/okay to test

@galipremsagar galipremsagar added the 5 - Ready to Merge Testing and reviews complete, ready to merge label Sep 25, 2024
@galipremsagar
Copy link
Contributor

/merge

@rapids-bot rapids-bot bot merged commit c1f377a into rapidsai:branch-24.10 Sep 25, 2024
98 checks passed
@lithomas1 lithomas1 deleted the pylibcudf-io-orc branch September 25, 2024 23:10
@lithomas1
Copy link
Contributor Author

Thanks for pushing this through!

@vyasr
Copy link
Contributor

vyasr commented Sep 27, 2024

Happy to! Thanks for getting it so far.

@Matt711 Matt711 mentioned this pull request Nov 20, 2024
Merged
3 tasks
rapids-bot bot pushed a commit that referenced this pull request Dec 9, 2024
@Matt711
Remove unused IO utilities from cudf python (#17374)
0f5d4b9 
Removes unused IO utilities from cuDF Python. Depends on #17163 #16042 #17252 #17263

Authors:
  - Matthew Murray (https://github.com/Matt711)

Approvers:
  - Bradley Dice (https://github.com/bdice)

URL: #17374
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vyasr vyasr vyasr left review comments

@galipremsagar galipremsagar galipremsagar approved these changes

@wence- wence- wence- approved these changes

@brandon-b-miller brandon-b-miller Awaiting requested review from brandon-b-miller brandon-b-miller is a code owner automatically assigned from rapidsai/cudf-python-codeowners

@bdice bdice Awaiting requested review from bdice

Assignees
No one assigned
Labels
5 - Ready to Merge Testing and reviews complete, ready to merge CMake CMake build issue feature request New feature or request non-breaking Non-breaking change pylibcudf Issues specific to the pylibcudf package Python Affects Python cuDF API.
Projects
cuDF Python
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@lithomas1 @vyasr @galipremsagar @wence- @bdice