Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pageserver: filter out the keys from parent shard but not belong to this shard when create delta layer #11018

Open
gripleaf opened this issue Feb 27, 2025 · 1 comment
Labels
external A PR or Issue is created by an external user

Comments

@gripleaf
Copy link

When use CompactionAlgorithm::Tiered to compact, and executing the CompactionStrategy::CreateDelta job, pageserver do not filter out the keys which from parent shard layer but not belong to this shard.

When create image, pageserver would filter out above keys.

code-create_delta

Is this code logic by design?

@github-actions github-actions bot added the external A PR or Issue is created by an external user label Feb 27, 2025
@jcsp
Copy link
Contributor

jcsp commented Mar 3, 2025

This is not by design. The Tiered compaction code was written around the same time that shard splitting was implemented, so they raced.

We are not currently using CompactionAlgorithm::Tiered for our production service, it should be regarded as experimental.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
external A PR or Issue is created by an external user
Projects
None yet
Development

No branches or pull requests

2 participants