Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sparse: add block-max wand and block-max maxscore algorithm #1026

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

sparknack
Copy link
Contributor

No description provided.

@sre-ci-robot sre-ci-robot requested review from hhy3 and liliu-z January 14, 2025 02:34
@sre-ci-robot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: sparknack
To complete the pull request process, please assign zhengbuqian after the PR has been reviewed.
You can assign the PR to them by writing /assign @zhengbuqian in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

mergify bot commented Jan 14, 2025

@sparknack 🔍 Important: PR Classification Needed!

For efficient project management and a seamless review process, it's essential to classify your PR correctly. Here's how:

  1. If you're fixing a bug, label it as kind/bug.
  2. For small tweaks (less than 20 lines without altering any functionality), please use kind/improvement.
  3. Significant changes that don't modify existing functionalities should be tagged as kind/enhancement.
  4. Adjusting APIs or changing functionality? Go with kind/feature.

For any PR outside the kind/improvement category, ensure you link to the associated issue using the format: “issue: #”.

Thanks for your efforts and contribution to the community!.

@zhengbuqian
Copy link
Collaborator

/hold

LOG_KNOWHERE_ERROR_ << "Failed to create mmap file when loading sparse InvertedIndex: " << strerror(errno);
return Status::disk_file_error;
}
temp_file.close();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I follow why this is needed. Is the goal to set the file size to zero?
What does this function do exactly?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, just create a empty file which will be used for inverted index mmap.

sort_cursors();
} else {
size_t next_list = pivot;
for (; cursor_ptrs[next_list]->cur_vec_id_ == pivot_id; --next_list) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it needed to add a 'out-of-bound' safeguard here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no need.
next_list will not smaller than 0 since it is guaranteed that pivot_id == cursor_ptrs[0]->cur_vec_id_ is not true.

Copy link

codecov bot commented Jan 23, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 73.61%. Comparing base (3c46f4c) to head (862f75a).
Report is 297 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff            @@
##           main    #1026       +/-   ##
=========================================
+ Coverage      0   73.61%   +73.61%     
=========================================
  Files         0       82       +82     
  Lines         0     7727     +7727     
=========================================
+ Hits          0     5688     +5688     
- Misses        0     2039     +2039     

see 82 files with indirect coverage changes

@mergify mergify bot added the ci-passed label Jan 23, 2025
@alexanderguzhva
Copy link
Collaborator

/lgtm overall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants