Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(reserve): evict just enough chunks to reach the capacity #4549

Merged
merged 4 commits into from
Jan 23, 2024

Conversation

istae
Copy link
Member

@istae istae commented Jan 17, 2024

Checklist

closes #4538

  • I have read the coding guide.
  • My change requires a documentation update, and I have done it.
  • I have added tests to cover my changes.
  • I have filled out the description and linked the related issues.

Description

Open API Spec Version Changes (if applicable)

Motivation and Context (Optional)

Related Issue (Optional)

Screenshots (if appropriate):

@istae istae changed the title fix: evict enough chunks only to fall below capacity fix(reserve): evict just enough chunks to reach the capacity Jan 17, 2024
@@ -335,6 +360,14 @@ func (r *Reserve) EvictBatchBin(
return 0, err
}

// evict oldest chunks first
sort.Slice(evicted, func(i, j int) bool {
return evicted[i].BinID < evicted[j].BinID
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Color me not fully grokking this, but how is sorting by BinID getting us the "oldest chunks first" as the comment states?

Copy link
Member Author

@istae istae Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

binID is an unique, incremented value that is assigned to each chunk in the reserve. So by removing the chunks with the lowest binID, we can at least bring some order (and not just randomly) as to which chunks should be removed first.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So binID has nothing to do with the pullsync bins, nor anything else within bee that is called bin? And since it is unique, I would gather that each "bin" contains but a single chunk? Bizarre.

One of my personal software development mantras is "Naming can make or break a project".

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reserve has "bins", which correspond to the proximity orders of the chunks (bin 0 = po 0, etc). so a "binID" is the unique ID of a chunk in the bin.

pullsync gets chunks from a peer using the peer's binIDs (so called bin intervals) for each bin/po.

@istae istae requested review from acha-bill and notanatol January 17, 2024 19:51
@istae istae changed the base branch from master to localstore-cleanup January 23, 2024 00:08
@istae istae force-pushed the localstore-cleanup branch from 32ec942 to 657f7ab Compare January 23, 2024 00:13
@istae istae merged commit 5e8f4a1 into localstore-cleanup Jan 23, 2024
12 checks passed
@istae istae deleted the evict-enough branch January 23, 2024 14:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

evict just enough chunks of a batch to fall below the reserve capacity
4 participants