Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[improve][test] Add solution to PulsarMockBookKeeper for intercepting reads #23875

Merged

Conversation

lhotari
Copy link
Member

@lhotari lhotari commented Jan 22, 2025

Motivation

In Pulsar tests, in-memory PulsarMockBookKeeper is used to implement BookKeeper client API.
There hasn't been a way to write realistic tests where there's variable latencies in BookKeeper reads.
Adding tests for various broker cache scenarios will benefit from this test framework improvement.

Example usage:

        // introduce delay to reads
        pulsarTestContext.getMockBookKeeper().setReadHandleInterceptor((ledgerId, firstEntry, lastEntry, entries) -> {
            log.info("intercepted read entries for ledgerId {}, firstEntry={}, lastEntry={}", ledgerId, firstEntry,
                    lastEntry);
            long delayMillis = ThreadLocalRandom.current().nextLong(25, 50);
            return CompletableFuture.supplyAsync(() -> entries,
                    CompletableFuture.delayedExecutor(delayMillis, TimeUnit.MILLISECONDS));
        });

The same approach can be used to inject failures.

Modifications

Add PulsarMockReadHandleInterceptor interface and implement it in PulsarMockBookKeeper.

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

@codecov-commenter
Copy link

codecov-commenter commented Jan 22, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 74.16%. Comparing base (bbc6224) to head (c937b12).
Report is 861 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff              @@
##             master   #23875      +/-   ##
============================================
+ Coverage     73.57%   74.16%   +0.58%     
+ Complexity    32624    31815     -809     
============================================
  Files          1877     1853      -24     
  Lines        139502   143619    +4117     
  Branches      15299    16311    +1012     
============================================
+ Hits         102638   106509    +3871     
+ Misses        28908    28727     -181     
- Partials       7956     8383     +427     
Flag Coverage Δ
inttests 26.81% <ø> (+2.23%) ⬆️
systests 23.18% <ø> (-1.14%) ⬇️
unittests 73.66% <ø> (+0.81%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1031 files with indirect coverage changes

@lhotari lhotari merged commit 87fb442 into apache:master Jan 23, 2025
61 of 62 checks passed
lhotari added a commit that referenced this pull request Jan 23, 2025
lhotari added a commit that referenced this pull request Jan 23, 2025
lhotari added a commit that referenced this pull request Jan 23, 2025
nikhil-ctds pushed a commit to datastax/pulsar that referenced this pull request Jan 31, 2025
… reads (apache#23875)

(cherry picked from commit 87fb442)
(cherry picked from commit 38f15bc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants