-
Notifications
You must be signed in to change notification settings - Fork 338
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: override same chunk #4749
fix: override same chunk #4749
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I like the look of this better, simply replace it within the transaction rather than the possibility of dangling/orphan chunks.
@@ -242,6 +242,13 @@ func (c *chunkStoreTrx) Iterate(ctx context.Context, fn storage.IterateChunkFn) | |||
return chunkstore.Iterate(ctx, c.indexStore, c.sharkyTrx, fn) | |||
} | |||
|
|||
func (c *chunkStoreTrx) Replace(ctx context.Context, ch swarm.Chunk) (err error) { | |||
defer handleMetric("chunkstore_replace", c.metrics)(&err) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See if I can remember to add this metric to one of my Grafana dashboards!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
While it resolves #4739 , I hoped we can handle two problems at the same time.
This fix does not cover the case when one uses postage stamp for the replaced single owner chunk; that stamp is used up for a non-living chunk on the uploader side when storage providers do not benefit from this pull syncing.
Former is not that significant problem especially if the batch is mutable while the last could be a huge improvement IMO since this is a supported feature in storage incentives with which operators can grow they reserve.
Nevertheless, there is no chunk retrieval mechanism to download different payloads of SOC at the moment, (also may require sharky changes) so we should merge this in order to have a more solid shelling point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
much better, let's also add some comments though
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, I was not aware of these kind of different stores used for storing chunk, but I have some questions/comments regarding that below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice!
one final change request https://github.com/ethersphere/bee/pull/4749/files#diff-e37eed2ab5dfe1533be0257473f5b76738361f02d2be71da1b7bf4d5216a00c7R294: the size change here now also has to consider the same chunk address check.
should be:
if !loadedStamp && sameAddressOldChunkStamp == nil {
r.size.Add(1)
}
Checklist
Description
Override chunk with the same address.
When we attempt to delete the chunk, the delete operation is appended to the current batch. However, since this Get reads from disk, it returns the old
refCnt
. So it ignores the new chunk.To fix this, we introduce a new
replace
method that does not relly onrefCnt
.Related Issue (Optional)
#4739