Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: minor refactorings around dense_set deletions #4390

Merged
merged 1 commit into from
Jan 2, 2025
Merged

Conversation

romange
Copy link
Collaborator

@romange romange commented Dec 31, 2024

Done as a preparation to introduce asynchronous deletions for sets/zsets/hmaps.

1. Restrict the interface around DbSlice::Del. Now it requires for the iterator to be valid and the checks should
   be explicit before the call. Most callers already provides a valid iterator.

2. Some minor refactoring in compact_object_test.
3. Expose DenseSet::ClearStep to allow iterative deletions of elements.

@@ -76,29 +76,38 @@ void DeallocateAtRandom(size_t steps, std::vector<void*>* ptrs) {
}
}

static void InitThreadStructs() {
auto* tlh = mi_heap_get_backing();
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

factored out the code below into dedicated functions. no changes here

@@ -168,14 +168,14 @@ auto DenseSet::PopPtrFront(DenseSet::ChainVectorIterator it) -> DensePtr {
return front;
}

uint32_t DenseSet::ClearInternal(uint32_t start, uint32_t count) {
uint32_t DenseSet::ClearStep(uint32_t start, uint32_t count) {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

renamed the function and made it public.

@romange romange changed the title chore: minor fixes around dense_set deletions chore: minor refactorings around dense_set deletions Dec 31, 2024
BorysTheDev
BorysTheDev previously approved these changes Jan 1, 2025
Done as a preparation to introduce asynchronous deletions for sets/zsets/hmaps.
1. Restrict the interface around DbSlice::Del. Now it requires for the iterator to be valid and the checks should
be explicit before the call. Most callers already provides a valid iterator.

2. Some minor refactoring in compact_object_test.
3. Expose DenseSet::ClearStep to allow iterative deletions of elements.

Signed-off-by: Roman Gershman <[email protected]>
Comment on lines +1041 to +1045
if (!IsValid(it))
continue;

db_slice.Del(op_args.db_cntx, it);
++res;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit

if (IsValid(it)) {
   db_slice.Del(op_args.db_cntx, it);
   ++res;
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

subjective :)

@adiholden
Copy link
Collaborator

Done as a preparation to introduce asynchronous deletions for sets/zsets/hmaps.

1. Restrict the interface around DbSlice::Del. Now it requires for the iterator to be valid and the checks should
   be explicit before the call. Most callers already provides a valid iterator.

2. Some minor refactoring in compact_object_test.
3. Expose DenseSet::ClearStep to allow iterative deletions of elements.

What is asynchronous deletions for sets/zsets/hmaps? how/when will it be used?
why do you need the change in Del intreface for this?

@romange
Copy link
Collaborator Author

romange commented Jan 2, 2025

@adiholden , asynchronous/iterative deletions of huge sets is similar to flushdb async : we delete the object in the background. See "UNLINK" for more details. I do not need to change DbSlice::Del for this, but I stumbled upon ambiguity in DbSlice::Del usage with 60% callers have CHECKS assuming the item exists but with some callers pass an invalid iterator leading to ambiguities like these: https://github.com/dragonflydb/dragonfly/pull/4390/files#diff-b08807737bb80c502bd26c2f2685e98e3c9299eaf38cae715852bd1c03730f18L446-L448

So I think its interface is better now and I wanted to separate these fixes from the async PR.

@@ -1169,10 +1169,10 @@ void HSetFamily::HRandField(CmdArgList args, const CommandContext& cmd_cntx) {
}
}

if (string_map->Empty()) {
auto it_mutable = db_slice.FindMutable(db_context, key, OBJ_HASH);
it_mutable->post_updater.Run();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you removed in few places in this pr calls to post_updater.Run()
and added a comment that it will run immediately.
how will it run? I believe that in the current interface we need to run it before we call del, not so good approach but I think that you break something here if you remove this

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ItAndUpdater is returned by FindMutable as a temporary and is destructed right after the call. ItAndUpdater object contains post_updater that will run during the destruction. It's not explicit. hence the comment that that explains it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now.. thats very confusing

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually borrowed it from Shachar's code from another place. I find it acceptable if it has a comment :)

@romange romange merged commit 7a68528 into main Jan 2, 2025
9 checks passed
@romange romange deleted the FixDel branch January 2, 2025 09:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants