Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[22321] Benchmark example updated #5460

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

Javgilavi
Copy link
Contributor

@Javgilavi Javgilavi commented Dec 4, 2024

Description

I remake the benchmark example to new example style maintaining the original functionality.

@Mergifyio backport 3.1.x 3.0.x

Fixes #5398

Contributor Checklist

  • Commit messages follow the project guidelines.
  • The code follows the style guidelines of this project.
  • Tests that thoroughly check the new feature have been added/Regression tests checking the bug and its fix have been added; the added tests pass locally
  • N/A Any new/modified methods have been properly documented using Doxygen.
  • N/A Any new configuration API has an equivalent XML API (with the corresponding XSD extension)
  • Changes are backport compatible: they do NOT break ABI nor change library core behavior.
  • Changes are API compatible.
  • New feature has been added to the versions.md file (if applicable).
  • N/A New feature has been documented/Current behavior is correctly described in the documentation.
  • Applicable backports have been included in the description.

Reviewer Checklist

  • The PR has a milestone assigned.
  • The title and description correctly express the PR's purpose.
  • Check contributor checklist is correct.
  • If this is a critical bug fix, backports to the critical-only supported branches have been requested.
  • Check CI results: changes do not issue any warning.
  • Check CI results: failing tests are unrelated with the changes.

@Javgilavi Javgilavi added this to the v3.2.0 milestone Dec 4, 2024
@Javgilavi Javgilavi requested a review from Mario-DL December 9, 2024 06:21
@github-actions github-actions bot added the ci-pending PR which CI is running label Dec 9, 2024
Copy link
Contributor

@rsanchez15 rsanchez15 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending example tests

@Javgilavi Javgilavi force-pushed the feature/benchmark-example branch from 4c8a8db to 54b8494 Compare December 11, 2024 08:48
@rsanchez15 rsanchez15 requested review from Mario-DL and removed request for Mario-DL December 11, 2024 09:58
@Javgilavi Javgilavi requested review from Mario-DL and removed request for Mario-DL December 11, 2024 12:40
Copy link
Member

@Mario-DL Mario-DL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good first iteration overall. The PR brings an interesting example.
In addition to my comments below, we would need to add this PR content Refactor benchmark example to the versions.md and replace the N/A with a tick in the checklist.

examples/cpp/benchmark/CLIParser.hpp Outdated Show resolved Hide resolved
examples/cpp/benchmark/CLIParser.hpp Outdated Show resolved Hide resolved
examples/cpp/benchmark/CLIParser.hpp Outdated Show resolved Hide resolved
examples/cpp/benchmark/CLIParser.hpp Outdated Show resolved Hide resolved
examples/cpp/benchmark/CLIParser.hpp Outdated Show resolved Hide resolved
Comment on lines 233 to 240
while (matched_ == 0)
{
std::unique_lock<std::mutex> initial_lock(mutex_);
auto check = cv_.wait_for(initial_lock, std::chrono::milliseconds(1), [&]()
{
return is_stopped();
});
Copy link
Member

@Mario-DL Mario-DL Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We typically recommend not to perform blocking operations in callbacks. I think it would make more sens moving these lines to the first part of the run() method.
Additionally, I think we could change the matched_ condition to be greater of equal to 2 since we expect both of out endpoints to be matched with the publisher ones.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is to make sure the subscriber not to sent back the data until the on_publication_match happen. To solve the case of the subscriber receiving the data but not sending it properly.
I don't understand how this could be moved to run().
Also, I understand the comment of change for matched_ to be greater or equal to 2 is for the publisher and not here, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The first thing in the run() method would be:

{
        // Wait for the data endpoints discovery
        std::unique_lock<std::mutex> matched_lock(mutex_);
        cv_.wait(matched_lock, [&]()
                {
                    // at least one has been discovered
                    return ((matched_ >= 2) || is_stopped());
                });
}

Then, I would remove all while (matched_ == 0){} blocks from the on_data_available(). This will make the subscriber to first wait to be matched and simplify the 'on_data_available()`. We should until greater or equal than 2 since both subscriber endpoints must be matched.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats more, we could avoid waiting for any matched_ within run(), since the reliability and durability will do the late-joining this for us when specified

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we discussed in private chat, I just delated the blocking operation on the on_data_available().

Comment on lines 74 to 82
Sample with index: '1' (8388608 Bytes) RECEIVED
Sample with index: '0' (8388608 Bytes) SENT
Sample with index: '1' (8388608 Bytes) RECEIVED
Sample with index: '0' (8388608 Bytes) SENT
Sample with index: '1' (8388608 Bytes) RECEIVED
Sample with index: '2' (8388608 Bytes) SENT
Sample with index: '3' (8388608 Bytes) RECEIVED
Sample with index: '4' (8388608 Bytes) SENT
Sample with index: '5' (8388608 Bytes) RECEIVED
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing locally I found a couple of strange things to me in the publisher side:

  • The first index seems to be sent and received twice
  • The RECEIVED is only printed for odd sample idexes (1,3,5,...)
    Is that behavior correct ?

Copy link
Contributor Author

@Javgilavi Javgilavi Dec 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The problem of repeating the first index is fix.
The RECEIVED is only printed for odd because they are sent by the subscriber and receive in the publisher, and the evens are sent by the publisher and receive by the subscriber

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I reviewed the logic and I first thought that the subscriber was not incrementing the index but it is doing it, so it is expected.

Copy link
Member

@Mario-DL Mario-DL Dec 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, update the commands output in this document since the index 0, index 1, index 0,... behavior was already corrected and no longer exists

examples/cpp/benchmark/README.md Outdated Show resolved Hide resolved
examples/cpp/benchmark/README.md Outdated Show resolved Hide resolved
test/examples/test_benchmark.py Outdated Show resolved Hide resolved
@Javgilavi Javgilavi force-pushed the feature/benchmark-example branch from b1e7b77 to 791e055 Compare December 16, 2024 06:28
@Javgilavi Javgilavi requested review from Mario-DL and rsanchez15 and removed request for Mario-DL December 16, 2024 06:28
@Javgilavi Javgilavi force-pushed the feature/benchmark-example branch from cc2b477 to f51d7b1 Compare December 16, 2024 08:56
@Javgilavi Javgilavi requested review from Mario-DL and removed request for Mario-DL December 16, 2024 08:57
Copy link
Member

@Mario-DL Mario-DL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Leaving just some comments on previous suggestions

@Mario-DL
Copy link
Member

Mario-DL commented Dec 16, 2024

Please, don't forget updating the versions.md and marking it in the contributor checklist

return;
}
}
if (benchmark_.index() >= samples_)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we are missing this here, otherwise when the example is launched without --samples the subscriber will receive the first sample and skip.

Suggested change
if (benchmark_.index() >= samples_)
if (samples_ != 0 && benchmark_.index() >= samples_)

@Javgilavi Javgilavi requested a review from Mario-DL December 16, 2024 11:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ci-pending PR which CI is running
Projects
None yet
Development

Successfully merging this pull request may close these issues.

no benchmark example? and incorrect doc instructions
3 participants