Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sortperm for simple vector #103

Open
gitboy16 opened this issue May 4, 2023 · 3 comments
Open

Sortperm for simple vector #103

gitboy16 opened this issue May 4, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@gitboy16
Copy link

gitboy16 commented May 4, 2023

Hi,
I was wondering if it possible to expose the parallel sortperm function for "normal" vectors like Vector{Float64} or Vector{Int64}?
Thank you
Kind regards

@sl-solution
Copy link
Owner

Probably not (it heavily depends on Dataset). I guess there should be other packages for parallel sorting, and I guess the Base sort should be quite fast for vector{T}. The real benefits of IMD are in the case

  • where missing values are exposed
  • or/and there are multiple columns
  • or/and there are many rows

which it will be worth to create a data set.

@gitboy16
Copy link
Author

gitboy16 commented May 4, 2023

Base sort is actually slow compare to other language like c++ boost sorting algorithms which are multi threaded. A parallel sortperm would be very useful for large vectors.

@sl-solution
Copy link
Owner

Base sort is actually slow compare to other language like c++ boost sorting algorithms which are multi threaded.

I see. When there is only one column, IMD uses a simple approach for parallel sorting, however for multiple columns and, particularly, in common scenarios of data manipulation tasks (and with QuickSort) it is much more efficient than other algorithms.

I guess, in general, we need some customised algorithms for Vector{T} before being able to expose sortperm to users.

PS I like to see support of other sorting algorithms in IMD (#47 ) and probably we can think about this during that time.

@sl-solution sl-solution added the enhancement New feature or request label May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants