Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][broker]Delete unnecessary calls #24045

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Conversation

guan46
Copy link
Contributor

@guan46 guan46 commented Mar 3, 2025

Motivation

There are unnecessary calls in the individualAckNormal. Delete the line of code that calls updateBlockedConsumerOnUnackedMsgs to improve performance.

Modifications

Refer to the Motivation section.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:guan46#8

@github-actions github-actions bot added the doc-not-needed Your PR changes do not impact docs label Mar 3, 2025
@lhotari
Copy link
Member

lhotari commented Mar 3, 2025

There are unnecessary calls in the individualAckNormal. Delete the line of code that calls updateBlockedConsumerOnUnackedMsgs to improve performance.

@guan46 Please explain what "improve performance" means. This type of PRs will be wasting reviewer's time when there isn't sufficient rationale. Instead of being helpful they become harmful. Generative AI could produce a huge amount of PRs for the Pulsar code base and put reviewers to train bots. That has happened in other OSS projects and we don't want that to happen in Apache Pulsar. Please pay more attention when you create new PRs. Adding a name to your profile could also help that we don't mix you into a bot account. Bot accounts will be blocked and reported.

@guan46
Copy link
Contributor Author

guan46 commented Mar 4, 2025

I've been learning about Pulsar recently. I learn it by reading PRs. If I find any issues, I'll also submit PRs myself. I'm not an AI. @lhotari

@lhotari
Copy link
Member

lhotari commented Mar 4, 2025

I've been learning about Pulsar recently. I learn it by reading PRs. If I find any issues, I'll also submit PRs myself. I'm not an AI. @lhotari

Thanks for your clarification, @guan46. Some Apache projects have been receiving PRs which are AI generated with human assistance and I wanted to check that. I hope you understand that's the reason why there's a pushback for anonymous contributors.

There's 2 concerns why projects avoid anonymous contributions: the trend of AI training by creating PRs and security. We also want to avoid security incidents where a contributor gains trust and later pushes malicious PRs that get accepted into code bases. The xz/liblzma backdoor incident is a sobering reminder of how anonymous trusted contributors can compromise critical infrastructure.

We can put security concerns aside. Since your profile was anonymous it was a red flag to me mainly related to the "AI training" scams that have been hitting OSS repositories. A real person opens the PR, but requested by a AI training company and the review feedback is fed back to train AI and to make follow up changes assisted by AI.
I see that you have added your name to your profile, which is very useful in avoiding such suspicions.

For this specific PR, it would be helpful if you could explain in more detail what performance improvements you expect from removing this call, perhaps with some measurements or profiling data that shows the impact. When you provide this clarification, reviewers won't have to spend their limited time in finding out these details.

For microbenchmarks we have the microbench module in Pulsar where new JMH microbenchmarks can be added. I'm not saying that it's necessary in this case if there's another way to explain what "improve performance" means and what the expected impact of the change is.

@guan46
Copy link
Contributor Author

guan46 commented Mar 4, 2025

In the individualAckNormal method, the checkCanRemovePendingAcksAndHandle method is called. (The checkCanRemovePendingAcksAndHandle method removes the pending acknowledgments and then updates the blocked state of the consumer.) After subsequently updating the number of unacknowledged messages, the updateBlockedConsumerOnUnackedMsgs method is called again to update the blocked state of the consumer. This is a redundant call. @lhotari

@lhotari
Copy link
Member

lhotari commented Mar 4, 2025

In the individualAckNormal method, the checkCanRemovePendingAcksAndHandle method is called. (The checkCanRemovePendingAcksAndHandle method removes the pending acknowledgments and then updates the blocked state of the consumer.) After subsequently updating the number of unacknowledged messages, the updateBlockedConsumerOnUnackedMsgs method is called again to update the blocked state of the consumer. This is a redundant call. @lhotari

This PR would introduce a bug in since not all locations of removePendingAcks are handled in this PR. I don't see that you have run tests in your own fork to check that all tests pass. It's possible that tests don't catch the problem that this PR in it's current form would introduce. Reviewing such PRs is time consuming without actual added value.

A recent change in this area was #23796 by @summeriiii. This information could be useful for other reviewers.

Copy link
Member

@lhotari lhotari left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please check comments

@guan46
Copy link
Contributor Author

guan46 commented Mar 15, 2025

The unit test has passed. Please take a look. @lhotari

@lhotari
Copy link
Member

lhotari commented Mar 15, 2025

The unit test has passed. Please take a look. @lhotari

@guan46 please confirm that you have considered all previous feedback and answer to the detailed points

@guan46
Copy link
Contributor Author

guan46 commented Mar 15, 2025

please confirm that you have considered all previous feedback and answer to the detailed points

Which question are you referring to that has not been answered @lhotari

@lhotari
Copy link
Member

lhotari commented Mar 15, 2025

This PR would introduce a bug in since not all locations of removePendingAcks are handled in this PR. I don't see that you have run tests in your own fork to check that all tests pass. It's possible that tests don't catch the problem that this PR in it's current form would introduce. Reviewing such PRs is time consuming without actual added value.

@guan46 please read #24045 (comment) carefully.

It would be great to find more useful contribution targets than eliminating unnecessary method calls for code locations which aren't hotspots confirmed with profiling. You can join the dev channel on Pulsar slack to find more valuable PR subjects from the Pulsar project's point of view.

@guan46
Copy link
Contributor Author

guan46 commented Mar 15, 2025

This PR would introduce a bug in since not all locations of removePendingAcks are handled in this PR. I don't see that you have run tests in your own fork to check that all tests pass. It's possible that tests don't catch the problem that this PR in it's current form would introduce. Reviewing such PRs is time consuming without actual added value.

@guan46 please read #24045 (comment) carefully.

It would be great to find more useful contribution targets than eliminating unnecessary method calls for code locations which aren't hotspots confirmed with profiling. You can join the dev channel on Pulsar slack to find more valuable PR subjects from the Pulsar project's point of view.

Has my PR made Pulsar better? For new members of the community, I believe that what's more important is to offer encouragement. I will grow step by step and make contributions to the community. @lhotari

@lhotari
Copy link
Member

lhotari commented Mar 15, 2025

Has my PR made Pulsar better? For new members of the community, I believe that what's more important is to offer encouragement. I will grow step by step and make contributions to the community. @lhotari

@guan46 This particular PR might introduce a bug, I mentioned that in the previous comment: "This PR would introduce a bug in since not all locations of removePendingAcks are handled in this PR." ... "It's possible that tests don't catch the problem that this PR in it's current form would introduce." . We don't have 100% test coverage and that's why doing this type of PRs is not a good way to start contributing.

It's better to address real issues that you notice when you use Pulsar, for example going through the user guide and noticing usability issues, inconsistency and bugs. There are plenty of issues that you will spot that way. For example, the Apache Pulsar Helm chart instructions aren't up-to-date in the user guide and there's a similar issue with docker and docker compose instructions that users will run into issues following the user guide. I'd prefer PRs that are addressing issues that you run into while using Pulsar yourself or within your company. Those improvements will surely be valuable for everyone.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
doc-not-needed Your PR changes do not impact docs
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants