Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fetch()/next() rely on missed heartbeats to detect abnormally ending next requests #150

Merged
merged 1 commit into from
Nov 20, 2024

Conversation

aricart
Copy link
Member

@aricart aricart commented Nov 20, 2024

fix(jetstream) fetch/next() now rely on missing heartbeat detection to fail requests that were not properly "finished" by the server. This has the benefit that messages arriving "late" are now likely to complete successfully since the client won't be proactively ending the request.

change(jetstream) non-ordered fetch/next will now throw/reject if heartbeats are missed - previously you could find out if you had statuses enabled on the fetch. On next() it was silently ignored and the client received a null message.

tests(jetstream): removed flapper tests that attempted yield ConsumerNotFound/StreamNotFound when a resource as removed prior to a consume/fetch/next.

tests(jestream): fixed cross-account configurations to have a response_type of "stream"

There's also a small implication for the edge case where a client is partitioned from jetstream towards the end of the fetch/next - in those cases is possible for the request to extend for up-to a minute (max heartbeat interval is 30s, and two misses are required). This is an edge condition when compared to the normal fetch/next that gets a new message towards the end of the expires. In this more common case, there won't be a redelivery since the message won't be dropped because of an overly eager timeout.

Fix #149

…to fail a requests that were not properly "finished" by the server. This has the benefit that messages arriving "late" are now likely to complete successfully since the client won't be proactively ending the request.

change(jetstream) non-ordered fetch/next will now throw/reject if heartbeats are missed - previously you could find out if you had statuses enabled on the fetch. On next() it was silently ignored and the client received a null message.

tests(jetstream): removed flapper tests that attempted yield ConsumerNotFound/StreamNotFound when a resource as removed prior to a consume/fetch/next.

tests(jestream): fixed cross-account configurations to have a response_type of "stream"

There's also a small implication for the edge case where a client is partitioned from jetstream towards the end of the fetch/next - in those cases is possible for the request to extend for up-to a minute (max heartbeat interval is 30s, and two misses are required). This is an edge condition when compared to the normal fetch/next that gets a new message towards the end of the expires. In this more common case, there won't be a redelivery since the message won't be dropped because of an overly eager timeout.

Fix #149

Signed-off-by: Alberto Ricart <[email protected]>
Copy link
Contributor

@piotrpio piotrpio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

When that's merged we should update the ADR and follow up in other clients.

@aricart aricart merged commit 8fe6b2c into main Nov 20, 2024
31 checks passed
@aricart aricart deleted the fix-149 branch November 20, 2024 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Messages from a pull consumer are not reliably received by consumer.next() when nats connection is slow.
2 participants