Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability for a Consumer to bypass retries and DLQ a terminally failing message #1167

Open
jcass8695 opened this issue Feb 6, 2024 · 3 comments

Comments

@jcass8695
Copy link

jcass8695 commented Feb 6, 2024

Is your feature request related to a problem? Please describe.
If I consume a message and fail during the processing of that message in a terminal way, I know that I don't want to retry it, I want to send it directly to the DLQ. Currently, if I wanted a persistent record of the message, I would either have to

  1. ACK the message and log it, write it to a database etc.
  2. NACK the message and retry it up to the configured max retries, when it would then land in the DLQ.

Describe the solution you'd like
I would like a consumer to be able to signal a message to be routed directly to the DLQ.

Describe alternatives you've considered
See options 1 & 2 above.

Additional context
Add any other context or screenshots about the feature request here.

@RobertIndie
Copy link
Member

I think we could add an interceptor in the DLQPolicy so that we can let users choose whether each message need to be retried. Does it make sense to you?

Before implementing this, we need to follow the PIP process and get approval. Are you willing to lead this discussion? Would you like to start by sending the discussion to the mailing list?
FYI, you can find the PIP process guide here: https://github.com/apache/pulsar/blob/master/pip/README.md

@jcass8695
Copy link
Author

I'm happy to lead the discussion on this. I agree that an interceptor in DLQPolicy looks to be a good route forward. I should have time to dedicate to it by the end of the week.

@jcass8695
Copy link
Author

PIP-367 PR created and sent to the mailing list for discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants