Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Introduce Wavelet Matrix #10

Merged
merged 81 commits into from
Aug 8, 2024
Merged

Introduce Wavelet Matrix #10

merged 81 commits into from
Aug 8, 2024

Conversation

Cydhra
Copy link
Owner

@Cydhra Cydhra commented Jul 16, 2024 •
edited
Loading

Merge the Wavelet Matrix (#4) into main. This Pull Request is tracking the progress.

Things left to do:

  • write tests for all convenience functions to make sure no oversights are left
  • write the quantile function to allow some summary statistics on the encoded sequence
  • mirror all rank/select functions to use a u64 in place of the BitVec parameter for convenient access with less than 64 bit alphabets
  • implement an iterator over the sequence
  • implement sequential prefix counting as a constructor (https://doi.org/10.1145/3457197)
  • implement FromIterator
  • finalize the decisions when to return None in checked rank/select functions, and when to panic
  • Add examples to doc comments
  • Add sorted iterator
  • predecessor and successor functions

@Cydhra Cydhra added the enhancement New feature or request label Jul 16, 2024
Cydhra added 18 commits July 17, 2024 01:19
@Cydhra
rank_range_u64 implementation
2a3d641
@Cydhra
rank_offset_u64, rank_u64 implementations and various fixes to doc co…
c4f8b67 
…mments and bounds checks
@Cydhra
_u64 select methods
cd5a401
@Cydhra
document mirrored api
549f52f
@Cydhra
implemented quantile function
4edadd0
@Cydhra
quantile documentation
fa1a642
@Cydhra
quantile_u64 and statistics methods
57dcd7a
@Cydhra
add workflow badge
c32d119
@Cydhra
replaced .len with .len() in iterator-macro, so the macro is less rel…
f8400bf 
…iant on the structure it is being used for
@Cydhra
moved util.rs into util folder to keep module code organized
9cff4d5
@Cydhra
extracted elias fano utils into submodule to keep code organized
bedab29
@Cydhra
extracted unroll macro into separate file
e528af9
@Cydhra
extracted general iter macros into submodule for organization
ae8c0c8
@Cydhra
renamed impl_bv_iter macro to impl_vector_iterator
6e50e5a
@Cydhra
vector macro now accepts alternative names for get and get_unchecked …
793c4e7 
…methods
@Cydhra
added parameter to vector macro to allow custom return types
4651612
@Cydhra
make sure into_iter calls also use the custom return type
548e876
@Cydhra
initial skeleton wavelet tree (construction is wrong)
cf921e7
Cydhra added 17 commits July 28, 2024 00:03
@Cydhra
added test for queries on empty matrices, fixed missing checks on som…
ba56401 
…e convenience functions
@Cydhra
fix: median function off-by-one, added test for bounds of convenience…
af29e13 
… functions
@Cydhra
fix: rank_range didnt check for empty range
8b6b0ea
@Cydhra
fix: empty intervals are a well-defined query and should return 0
4b42ce6
@Cydhra
removed todos. We keep returning None on u64-queries on matrices with…
e74cc02 
… longer elements
@Cydhra
test for select bounds
fa36c60
@Cydhra
implemented predecessor function
6c31761
@Cydhra
fix: predecessor calls could fail if the range it keeps track of is e…
ee6c74e 
…mpty (i.e. if there is no predecessor)
@Cydhra
implemented successor function
ea2a598
@Cydhra
fix: only update next_greater_interval if we haven't moved past the l…
cd2bee8 
…ast eligible element already
@Cydhra
fix: large gap pred/succ queries fixed.
30a043b 
The prefix could be malformed due to a longer tree traversal until we notice that our interval doesnt contain the result, and those changes to the prefix must be reverted. Now the pred/succ function just clones the prefix at the level where we start quantile search from, so the prefix is guaranteed correct.

The code for pred/succ and their _u64 counterparts is also deduplicated
@Cydhra
the writer functions don't have mutable receivers, so they dont need …
e1f7a45 
…to be FnMut
@Cydhra
cleanup: use function-pointers instead of closures where appropriate
551bf7d
@Cydhra
documentation about sources and complexity
56dcdcf
@Cydhra
implemented sorted iterators
2aa8060
@Cydhra
added fuzzing test for pred/succ values
96ff46e
@Cydhra
fix: when switching the last branching interval in pred/succ search, …
af2c8dc 
…the prefix must be updated with the corresponding bit
@Cydhra Cydhra marked this pull request as draft July 28, 2024 16:18
@Cydhra Cydhra linked an issue Aug 8, 2024 that may be closed by this pull request
WaveletMatrix #4
Closed
@Cydhra Cydhra marked this pull request as ready for review August 8, 2024 14:48
@Cydhra Cydhra merged commit 479105b into master Aug 8, 2024
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

WaveletMatrix
1 participant
@Cydhra