Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storages replication doesn't replicate the deletions of key expressions #713

Open
JEnoch opened this issue Feb 2, 2024 · 2 comments
Open
Labels
bug Something isn't working

Comments

@JEnoch
Copy link
Member

JEnoch commented Feb 2, 2024

Describe the bug

If 2 aligned storages suffer a partition and during this partition some key expressions are deleted, after re-alignment of storages the deleted keys are still present in the replicated storage.

To reproduce

Install zenohd and the FileSystem backend

  1. Download and unzip del_test.zip
  2. Run del_test.sh.
    This test run 2 zenohd with replicated filesystem storages. It publishes 4 k/v that are stored and replicated, kills the 2nd zenohd, deletes 2k/v, restarts the 2nd k/v and waits for alignment of deleted keys which never happens

While the storage in ~/.zenoh/zenoh_backend_fs/d1 has only 2 remaining files, the storage in ~/.zenoh/zenoh_backend_fs/d2 always has 4 files.

System info

  • Zenoh v0.10.0-rc
@JEnoch JEnoch added the bug Something isn't working label Feb 2, 2024
@JEnoch
Copy link
Member Author

JEnoch commented Feb 2, 2024

The storages replication relies on Zenoh queries: each participant requesting alignment information and missing data to the others.
However, in current version, a Zenoh queryable can't send a reply with a "deleted" Sample. Even if the API accepts a Sample object in Query::reply(Sample) operation and this Sample contains a SampleKind::Delete, the Reply protocol message has no place for this SampleKind.

This issue will be addressed in #673 (checkbox "SampleKind in replies")

Then the align queryable shall be modified to:

  1. Retrieve from its storage not only the existing key values, but also the deleted ones from tombstone file. Currently a Session::get() is used (without SessionLocal destination!!). Would be better to directly query the StorageService object.
  2. Reply also deleted samples here

@Mallets
Copy link
Member

Mallets commented Feb 22, 2024

An update of the protocol is being developed in #protocol_change branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

2 participants