Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[2SP] CMW: Port everything that can be to native 0qm subscriptions #337

Open
ivan-cukic opened this issue Dec 21, 2023 · 1 comment
Open
Assignees

Comments

@ivan-cukic
Copy link
Collaborator

ivan-cukic commented Dec 21, 2023

Check if we're using old subscription mechanisms in OpenCMW. Wherever possible, remove use of SUBSCRIBE/UNSUBSCRIBE via router socket (mdp://) by zmq's native PUB/SUB mechanism. Document in case there are any limitations to PUB/SUB that require the old mechanism to be kept. Adjust unit tests.

For the REST backend, there is an opened PR that needs updating
#277

@ivan-cukic ivan-cukic converted this from a draft issue Dec 21, 2023
@RalphSteinhagen RalphSteinhagen moved this from 🆕 Ideas (∞) to 📋 Backlog in Digitizer Reimplementation Dec 21, 2023
@frankosterfeld frankosterfeld self-assigned this Feb 26, 2025
@RalphSteinhagen RalphSteinhagen changed the title CMW: Port everything that can be to native 0qm subscriptions [2SP] CMW: Port everything that can be to native 0qm subscriptions Feb 26, 2025
@RalphSteinhagen RalphSteinhagen moved this from 📋 Backlog to 🔖 Selected (6) in Digitizer Reimplementation Feb 26, 2025
@frankosterfeld frankosterfeld moved this from 🔖 Selected (6) to 🏗 In progress in Digitizer Reimplementation Feb 26, 2025
@frankosterfeld
Copy link
Contributor

frankosterfeld commented Feb 27, 2025

I've found a limitation when using the native 0mq PUB/SUB mechanism between clients and a broker.

Let's assume three clients A, B, C subscribe to A: /colors, B: /colors?red, C: /colors?green&red. (The params being alphabetically sorted is enforced by the openCMW API).

The worker publishes a notification with topic /colors?contentType=application%2Fjson&green&red, which matches all three subscriptions from A, B, C. The broker can skip publishing it with /colors, as the 0mq subscription message ensures that a message to /colors?red or /colors?green&red is also received by A.

But that's the problem: The broker must publish the message to both /colors?red and colors?green&red, to ensure that both B and C receive the message.

But as A's subscription to /colors matches both, A will receive the message twice.

I'm reading that RADIO/DISH uses exact matching, I can test that. It's in draft state though.

One could also hack around the prefix matching but appending some reserved "end token" to each subscription to prevent prefix matching to ever succeed. (/colors/$END$, /colors?red$END$ or similar)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🏗 In progress
Development

No branches or pull requests

3 participants