Java: Add ExamplesApp and Update benchmarkingApp to both have Java/Glide-for-redis client #896

acarbonetto · 2024-02-02T07:20:55Z

Issue #, if available:

Description of changes:

Add the Glide async client to the BenchmarkApp for the install_and_test script.
Add the ExamplesApp for manually testing custom commands.

Out of scope

Glide async client doesn't have cluster-mode client linked (TODO).
ExamplesApp runs only custom command (no transactions, no specific commands - TODO).

Examples:

To run benchmark app, run:

./gradlew :benchmarks:run --args="-clients GLIDE -dataSize 100 -concurrentTasks 100 -clientCount 1"

To run examples app, run:

./gradlew :examples:run

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

java/README.md

java/benchmarks/build.gradle

Yury-Fridlyand · 2024-02-02T19:07:22Z

java/benchmarks/src/main/java/glide/benchmarks/BenchmarkingApp.java

                    System.out.println("Run LETTUCE async client");
                    testClientSetGet(LettuceAsyncClient::new, runConfiguration, true);
                    break;
                case GLIDE:
-                    System.out.println("GLIDE for Redis async not yet configured");
+                    System.out.println("Run GLIDE async client");


woohoo
does it run in CI?

No. Does it need to?

https://github.com/aws/glide-for-redis/blob/main/.github/workflows/java-benchmark.yml
It runs on demand only. Can you trigger it and validate that it works?

https://github.com/Bit-Quill/glide-for-redis/actions/runs/7841103142
fails to compile using protoc.

Oh you need to add protobuf installation to benchmarking CI
Please update it to let CI pass and generate numbers.

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClient.java

java/examples/build.gradle

java/examples/src/main/java/glide/examples/ExamplesApp.java

java/examples/src/main/java/glide/examples/clients/GlideClient.java

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClient.java

shachlanAmazon · 2024-02-04T07:45:28Z

java/examples/build.gradle

@@ -0,0 +1,25 @@
+plugins {


examples should be under the ${root}/examples folder, not under the java folder. This reduces the user's ability to find the app.

This adds difficulties on dependency resolution (line 12), until java client released to maven.
But I think it is possible to commit a directory symlink. @acarbonetto can you try this?

I will try.
Ultimately, we want to be able to create a distribution JAR, and at that point a separate gradle project may be fine.

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClient.java

shachlanAmazon · 2024-02-04T07:51:51Z

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClient.java

+
+    @Override
+    public CompletableFuture<String> asyncSet(String key, String value) {
+        return redisClient.customCommand(new String[] {"Set", key, value}).thenApply(r -> "Ok");


why do you modify the returned value?

Previous iterations of the client returned null.
This is no longer true, and now the client returns "OK".
Fixed.

java/README.md

shachlanAmazon · 2024-02-04T13:36:35Z

I pulled this PR, ran ./install_and_test.sh -java -only-glide -tasks 500 -data 100 -no-tls and got

> Task :benchmarks:run FAILED
Run GLIDE async client
Exception in thread "main" java.lang.UnsatisfiedLinkError: no glide_rs in java.library.path: [../target/release]
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2678)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
        at java.base/java.lang.System.loadLibrary(System.java:1886)
        at glide.ffi.resolvers.SocketListenerResolver.<clinit>(SocketListenerResolver.java:11)
        at glide.api.BaseClient.buildChannelHandler(BaseClient.java:84)
        at glide.api.BaseClient.CreateClient(BaseClient.java:50)
        at glide.api.RedisClient.CreateClient(RedisClient.java:28)
        at glide.benchmarks.clients.glide.GlideAsyncClient.connectToRedis(GlideAsyncClient.java:32)
        at glide.benchmarks.utils.Benchmarking.testClientSetGet(Benchmarking.java:118)
        at glide.benchmarks.BenchmarkingApp.main(BenchmarkingApp.java:59)

FAILURE: Build failed with an exception.

acarbonetto · 2024-02-09T07:41:24Z

I pulled this PR, ran ./install_and_test.sh -java -only-glide -tasks 500 -data 100 -no-tls and got

> Task :benchmarks:run FAILED
Run GLIDE async client
Exception in thread "main" java.lang.UnsatisfiedLinkError: no glide_rs in java.library.path: [../target/release]
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2678)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:830)
        at java.base/java.lang.System.loadLibrary(System.java:1886)
        at glide.ffi.resolvers.SocketListenerResolver.<clinit>(SocketListenerResolver.java:11)
        at glide.api.BaseClient.buildChannelHandler(BaseClient.java:84)
        at glide.api.BaseClient.CreateClient(BaseClient.java:50)
        at glide.api.RedisClient.CreateClient(RedisClient.java:28)
        at glide.benchmarks.clients.glide.GlideAsyncClient.connectToRedis(GlideAsyncClient.java:32)
        at glide.benchmarks.utils.Benchmarking.testClientSetGet(Benchmarking.java:118)
        at glide.benchmarks.BenchmarkingApp.main(BenchmarkingApp.java:59)

FAILURE: Build failed with an exception.

It didn't build the rustn library. I have added a dependency on the run task to compile rust release first. This should now work out of the box.

I ran:

glide/benchmarks> ./install_and_test.sh -java -only-glide -tasks 100 -data 100 -no-tls

and got:

> Task :client:buildRustRelease
    Finished release [optimized + debuginfo] target(s) in 0.19s

> Task :benchmarks:run
GLIDE for Redis async client
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

 =====> Glide Async <===== 1 clients 100 concurrent 100 data size 

Runtime s: 5.473331
Iterations: 1000000
TPS: 182704.083490
===> SET <===
avg. time ms: 0.000544
std dev ms: 0.000244
p50 latency ms: 0.000518
p90 latency ms: 0.000591
p99 latency ms: 0.000994
Total hits: 200212
===> GET_NON_EXISTING <===
avg. time ms: 0.000544
std dev ms: 0.000250
p50 latency ms: 0.000518
p90 latency ms: 0.000591
p99 latency ms: 0.000990
Total hits: 159560
===> GET_EXISTING <===
avg. time ms: 0.000545
std dev ms: 0.000256
p50 latency ms: 0.000518
p90 latency ms: 0.000591
p99 latency ms: 0.000999
Total hits: 640228
Total hits: 1000000


> Task :examples:run
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Glide PING: PONG
Glide PING: found you

...note: I'll remove the examples:run :D from the install_and_test script.

.github/workflows/java-benchmark.yml

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClusterClient.java

Yury-Fridlyand · 2024-02-09T17:54:37Z

java/examples/build.gradle

@@ -0,0 +1,25 @@
+plugins {


This adds difficulties on dependency resolution (line 12), until java client released to maven.
But I think it is possible to commit a directory symlink. @acarbonetto can you try this?

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClient.java

java/README.md

java/benchmarks/src/main/java/glide/benchmarks/clients/glide/GlideAsyncClient.java

java/examples/src/main/java/glide/examples/ExamplesApp.java

java/examples/src/main/java/glide/examples/clients/GlideClient.java

Signed-off-by: Andrew Carbonetto <[email protected]>

Signed-off-by: Yury-Fridlyand <[email protected]>

Signed-off-by: Andrew Carbonetto <[email protected]>

Yury-Fridlyand · 2024-02-15T00:36:50Z

java/examples/src/main/java/glide/examples/ExamplesApp.java

+
+            System.out.println("SET(apples, oranges): " + client.set("apples", "oranges").get());
+            System.out.println("GET(apples): " + client.get("apples").get());
+


Is it worth adding an example with an error?

acarbonetto added the java issues and fixes related to the java client label Feb 2, 2024

acarbonetto self-assigned this Feb 2, 2024

acarbonetto requested a review from a team as a code owner February 2, 2024 07:20

acarbonetto requested a review from Yury-Fridlyand February 2, 2024 07:51

acarbonetto mentioned this pull request Feb 2, 2024

Add examples for java clients Bit-Quill/valkey-glide#53

Closed

Yury-Fridlyand reviewed Feb 2, 2024

View reviewed changes

shachlanAmazon reviewed Feb 4, 2024

View reviewed changes

acarbonetto closed this Feb 9, 2024

acarbonetto force-pushed the java/integ_acarbo_add_examples_benchmarking branch from ceb0144 to ceaa793 Compare February 9, 2024 07:13

acarbonetto reopened this Feb 9, 2024

Yury-Fridlyand reviewed Feb 9, 2024

View reviewed changes

shachlanAmazon reviewed Feb 11, 2024

View reviewed changes

shachlanAmazon approved these changes Feb 12, 2024

View reviewed changes

acarbonetto and others added 8 commits February 12, 2024 21:30

Re-add examples and benchmarking app

429c5a8

Signed-off-by: Andrew Carbonetto <[email protected]>

Spotless

536af94

Signed-off-by: Andrew Carbonetto <[email protected]>

Trigger java-benchmark

2b56ee7

Signed-off-by: Andrew Carbonetto <[email protected]>

Update benchmark CI.

be73ab7

Signed-off-by: Yury-Fridlyand <[email protected]>

Run benchmarks:run

c1f83a7

Signed-off-by: Andrew Carbonetto <[email protected]>

Update README examples

ca8dda5

Signed-off-by: Andrew Carbonetto <[email protected]>

Simplify ExampleApp

ad10c52

Signed-off-by: Andrew Carbonetto <[email protected]>

Combine clients for benchmarking

2259992

Signed-off-by: Andrew Carbonetto <[email protected]>

acarbonetto force-pushed the java/integ_acarbo_add_examples_benchmarking branch from 2fa01c2 to 2259992 Compare February 13, 2024 17:47

Remove unused files

558fde4

Signed-off-by: Andrew Carbonetto <[email protected]>

Yury-Fridlyand approved these changes Feb 13, 2024

View reviewed changes

Add set/get to examples app

f836c96

Signed-off-by: Andrew Carbonetto <[email protected]>

Yury-Fridlyand mentioned this pull request Feb 14, 2024

Java: Add UDS read/write error handling and tests with glide core mock #949

Merged

Add timeouts to benchmark client create

d1c4ed0

Signed-off-by: Andrew Carbonetto <[email protected]>

Yury-Fridlyand reviewed Feb 15, 2024

View reviewed changes

acarbonetto merged commit 7292960 into valkey-io:main Feb 15, 2024
11 checks passed

acarbonetto deleted the java/integ_acarbo_add_examples_benchmarking branch February 15, 2024 16:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Java: Add ExamplesApp and Update benchmarkingApp to both have Java/Glide-for-redis client #896

Java: Add ExamplesApp and Update benchmarkingApp to both have Java/Glide-for-redis client #896

acarbonetto commented Feb 2, 2024

Yury-Fridlyand Feb 2, 2024

acarbonetto Feb 2, 2024

Yury-Fridlyand Feb 2, 2024

acarbonetto Feb 9, 2024 •

edited

Loading

Yury-Fridlyand Feb 9, 2024

shachlanAmazon Feb 4, 2024

Yury-Fridlyand Feb 9, 2024

acarbonetto Feb 11, 2024

shachlanAmazon Feb 4, 2024

acarbonetto Feb 9, 2024

shachlanAmazon commented Feb 4, 2024

acarbonetto commented Feb 9, 2024 •

edited

Loading

Yury-Fridlyand Feb 9, 2024

Yury-Fridlyand Feb 15, 2024


		System.out.println("SET(apples, oranges): " + client.set("apples", "oranges").get());
		System.out.println("GET(apples): " + client.get("apples").get());

Java: Add ExamplesApp and Update benchmarkingApp to both have Java/Glide-for-redis client #896

Java: Add ExamplesApp and Update benchmarkingApp to both have Java/Glide-for-redis client #896

Conversation

acarbonetto commented Feb 2, 2024

Out of scope

Examples:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acarbonetto Feb 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shachlanAmazon commented Feb 4, 2024

acarbonetto commented Feb 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acarbonetto Feb 9, 2024 •

edited

Loading

acarbonetto commented Feb 9, 2024 •

edited

Loading