Client interests #863
Merged
Client interests #863
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@OlivierHecart If this pull request contains a bugfix or a new feature, then please consider using |
jean-roland
approved these changes
Mar 27, 2024
OlivierHecart
added a commit
that referenced
this pull request
May 13, 2024
…improve scalability (#814) * Router implements interests protocol for clients * Send WireExpr in UndeclareSubscriber/UndeclareQueryable to clients for pico * Fix WireExprExt M flag encoding/decoding * Fix decl_key * Clients send all samples and queries to routers and peers * Avoid self declaration loop on interest * Fix query/replies copy/paste bugs * Peers implement interests protocol for clients * Don't send WireExpr in UndeclareSubscriber/UndeclareQueryable to clients * Add client writer-side filtering (#863) * Add client writer-side filtering * Reimplement liveliness with interests * Fix writer-side filtering before receiving FinalInterest * Fix pubsub interest based routing after router failover * Declare message can be Push/Request/RequestContinuous/Response * Address review comments * Remove F: Future flag from DeclareInterest * cargo fmt --all * Remove unused Interest flags field * Update doc * Remove unneeded interest_id field * Update commons/zenoh-protocol/src/network/declare.rs * Remove unused UndeclareInterest * Implement proper Declare Request/Response id correlation * Add new Interest network message * Update doc * Update codec * Fix stable build * Fix test_acl * Fix writer side filtering * Add separate functions to compute matching status * Fix unstable imports * Remove useless checks --------- Co-authored-by: Luca Cominardi <[email protected]>
OlivierHecart
added a commit
that referenced
this pull request
May 15, 2024
* Router implements interests protocol for clients * Send WireExpr in UndeclareSubscriber/UndeclareQueryable to clients for pico * Fix WireExprExt M flag encoding/decoding * Fix decl_key * Clients send all samples and queries to routers and peers * Avoid self declaration loop on interest * Fix query/replies copy/paste bugs * Peers implement interests protocol for clients * Don't send WireExpr in UndeclareSubscriber/UndeclareQueryable to clients * Add client writer-side filtering (#863) * Add client writer-side filtering * Reimplement liveliness with interests * Fix writer-side filtering before receiving FinalInterest * Fix pubsub interest based routing after router failover * Declare message can be Push/Request/RequestContinuous/Response * Address review comments * Remove F: Future flag from DeclareInterest * cargo fmt --all * Remove unused Interest flags field * Update doc * Remove unneeded interest_id field * Update commons/zenoh-protocol/src/network/declare.rs * Remove unused UndeclareInterest * Implement proper Declare Request/Response id correlation * Add new Interest network message * Update doc * Update codec * Fix stable build * Fix test_acl * Fix writer side filtering * Add separate functions to compute matching status * Fix unstable imports * Remove useless checks * Don't apply open session delay in client mode * Add open_delay test * Peers don't apply writer side filtering until FinalInterest is received * Don't wait for full scouting delay when peers connected all configured connect endpoints * Increase scouting delay and decrease api open delay * Wait for gossip and related connections attempts before returning to open * Remove random backoff for p2p * Fix memory leak * Remove API_OPEN_DELAY * Don't apply any scouting delay when multicast disabled and no configured connect/endpoints * Sleep for scouting/delay in router and linkstate peer modes --------- Co-authored-by: Luca Cominardi <[email protected]>
OlivierHecart
added a commit
that referenced
this pull request
Jun 18, 2024
* Router implements interests protocol for clients * Send WireExpr in UndeclareSubscriber/UndeclareQueryable to clients for pico * Fix WireExprExt M flag encoding/decoding * Fix decl_key * Clients send all samples and queries to routers and peers * Avoid self declaration loop on interest * Fix query/replies copy/paste bugs * Peers implement interests protocol for clients * Don't send WireExpr in UndeclareSubscriber/UndeclareQueryable to clients * Add client writer-side filtering (#863) * Add client writer-side filtering * Reimplement liveliness with interests * Fix writer-side filtering before receiving FinalInterest * Fix pubsub interest based routing after router failover * refactor: Add boilerplace for new liveliness router hat * refactor: Handle incoming liveliness token declaration/undeclaration * refactor: Take `TokenId` id in `HatLivelinessTrait` methods * refactor: Implement router `HatLivelinessTrait` * chore: Add copyright headers * refactor: Implement client `HatLivelinessTrait` * refactor: Implement liveliness dispatching logic * refactor: Use `DeclareToken`/`UndeclareToken` in liveliness declaration/undeclaration * refactor: Implement Token Interest protocol for client & router * fix: Use correct token state in Liveliness query replies * chore: Temporarily Allow dead_code and unused_variables in p2p_peer and linkstate_peer hat * style: Rename `Primitives` to `IngressPrimitives` and `EPrimitives` to `EgressPrimitives` Ingress means "entering the router" and Egress means "leaving the router". This change could of course be reverted (through r-a) right before merging the refactor/liveliness-declaration branch. It would've been even better to move Session's IngressPrimitives impl to an EgressPrimitives, but that's not possible since a Session can recieve Query replies from itself. Ideally, Session in/egress should be Router e/ingress but the `EPrimitives` trait breaks this symmetry. Messages leaving the router are not the same as messages leaving the Session. This could be solved by making the `EgressPrimitives` trait generic over the message context, but the necessary refactoring would be a hude undertaking, and I don't have enough time nor familiarity with the codebase for it. * ci: Fix `clippy::suspicious_open_options` lint * chore: Fix naming issue after rebasing against `interests` * style: Rename liveliness to token * refactor: Add p2p_peer token implementation * fix: Set token id to 0 in multicast token declaration * fix: Mark unused arguments in p2p_peer `HatTokenTrait` impl * refactor: Add linkstate_peer token implementation * fix: Declare `Interest::TOKEN` in liveliness subscriber declaration * wip: Implement liveliness subscriber without liveliness prefix * Declare message can be Push/Request/RequestContinuous/Response * Address review comments * Remove F: Future flag from DeclareInterest * cargo fmt --all * Remove unused Interest flags field * Update doc * Remove unneeded interest_id field * Update commons/zenoh-protocol/src/network/declare.rs * Remove unused UndeclareInterest * Implement proper Declare Request/Response id correlation * Add new Interest network message * Update doc * Update codec * Fix stable build * Fix test_acl * Fix writer side filtering * fix: Cleanup token ressources * fix: Remove multicast token propagation * fix: Logic error in `Session::undeclare_subscriber_inner` * fix: Change `log::debug` to `log::trace` in `IngressPrimitives` * revert: Restore names of `Primitives` and `EPrimitives` methods * Add separate functions to compute matching status * Fix unstable imports * Remove useless checks * fix: Correctly set `interest_id` field everywhere * feat: Implement liveliness queries in `Session` * fix: Discard liveliness query on `DeclareFinal` with known interest id * feat: Propagate client liveliness queries, send local replies * feat: Handle liveliness query (final) replies * fix: Apply Clippy lints from Rust 1.77 * fix: `RwLock` deadlock in `Session::undeclare_subscriber_inner` * fix: Remove `zenoh::net::routing::PREFIX_LIVELINESS` The key assumption here is that no key expression can start with '@/liveliness' anymore. * fix: Set `ext_qos` to `QoSType::DEFAULT` when for liveliness queries * fix: Don't immediatly reply with `DeclareFinal` to token interest * fix: Send `DeclareToken` w/ interest id for `InterestMode::Current` * fix: Send `DeclareFinal` in response to `Current` token interest * fix: Set `ext_qos: QoSType::DEFAULT` in routing interest declaration * fix: Share ownership of `TokenQuery` b/w destination faces * fix: Remove unused code * fix: Rustfmt errors with CI config * test: Liveliness clique/brokered and subscriber/query scenarios * fix: Don't declare tokens with interest ids for `CurrentFuture` * fix: Incorrect wire expr for liveliness undeclaration callbacks * fix: Support liveliness queries to linkstate/p2p peers * test: Querying & fetching liveliness subscribers * fix: Incorrect keyexpr in liveliness query replies * fix: Remove `LivelinessQueryState::key_expr` * Fix locking * Only store tokens in response of a Future interest * Rename interest_id_counter * Remove dataroute related code from token dispatcher * Fix client token interest local replies * Remove no more needed compute_local_replies * Move TokenQuery to token module * Fix details * Use proper id in DeclareToken for p2p_peer * Remove comments * Add needed token_new_face functions * Don't register declares sent as response to a current interest * Code reorg * Add missing token_remove_node, token_tree_change and token_linkstate_change * Fix querying subscriber liveliness test * Send one shot Undeclares * Fix clippy warnings * Add misssing token related code in close_face * Properly propagate one shot undeclare token * Fix querying subscriber liveliness test * Fix invalid interest aggregate option handling * Fix interest propagation when pub client connecting to peer * Code reorg * Peers wait for DeclareFinal from routers before propagating to clients * Fix InterestFinal propagation * Only send back DeclareFinal if interest is current * Terminate liveliness interest for subsystems of peers * Fix liveliness test includes * Add liveliness local tests * Address review comments * Fix compilation * Fix stable build * Apply 2024 copyright to new files (interests new files as well) * Avoid clones * Remove unwraps * Fix Value usage after merge * Remove useless checks * Change &Option<T> tp Option<&T> * Revert wrongly commited change in z_liveliness example --------- Co-authored-by: OlivierHecart <[email protected]> Co-authored-by: OlivierHecart <[email protected]> Co-authored-by: Luca Cominardi <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.