Introduce Prometheus Java Client integration test #1468

rickie · 2024-12-18T13:30:48Z

There are some XXX's left to resolve, but apart from those, I think the PR is ready to be reviewed!

For those XXX's I'll still work on creating a PR for them.

github-actions · 2024-12-18T13:33:03Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-18T13:33:15Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-19T09:08:39Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

rickie · 2024-12-19T09:36:23Z

The IT is successful!

github-actions · 2024-12-19T13:51:48Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

rickie · 2024-12-19T15:50:27Z

integration-tests/prometheus-java-client-init.patch

+
+ class PrometheusInstrumentationScopeTest {
+
+  // XXX: Introduce support for rewriting `assertThatThrownBy#havingRootCause()`.


Filed: #1471

github-actions · 2024-12-20T13:43:59Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-20T13:45:14Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202

Rebased and added a commit. One point of attention: there appears to be a flaky test:

[ERROR] io.prometheus.metrics.core.metrics.SlidingWindowTest.rotate -- Time elapsed: 0.023 s <<< FAILURE!
org.opentest4j.AssertionFailedError:
[Start time: 1734782202868, current time: 1734782232878, elapsed time: 30010]
expected: [1.0, 1.0, 1.0, 1.0, 1.0]
 but was: [1.0, 1.0, 1.0, 1.0]
    at io.prometheus.metrics.core.metrics.SlidingWindowTest$Observer.assertValues(SlidingWindowTest.java:30)
    at io.prometheus.metrics.core.metrics.SlidingWindowTest.rotate(SlidingWindowTest.java:57)
    at java.base/java.lang.reflect.Method.invoke(Method.java:569)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
    at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)

@mohamedsamehsalah if you review #1471 then we can strike one precondition off the list :)

Stephan202 · 2024-12-21T11:04:00Z

integration-tests/prometheus-java-client.sh

+# XXX: Minimize the diff by including
+# `-XepOpt:Slf4jLoggerDeclaration:CanonicalStaticLoggerName=logger` once such
+# flags are supported in patch mode. See
+# https://github.com/google/error-prone/pull/4699.


This comment doesn't apply here 👀

Stephan202 · 2024-12-21T11:07:54Z

integration-tests/prometheus-java-client-init.patch

+         <junit-jupiter.version>5.11.3</junit-jupiter.version>
+         <otel.instrumentation.version>2.10.0-alpha</otel.instrumentation.version>
+-        <java.version>8</java.version>
+        <java.version>11</java.version>


We can specify this using additional_build_flags='-Djava.version=11' instead, keeping the patch minimal.

Stephan202 · 2024-12-21T11:11:21Z

integration-tests/prometheus-java-client-init.patch

+                     <showWarnings>true</showWarnings>
+                     <compilerArgs>
+                         <arg>-Xlint:all,-serial,-processing,-options</arg>
+-                        <arg>${warnings}</arg>


We can instead set -Dwarnings=, keeping the patch minimal.

Stephan202 · 2024-12-21T11:44:29Z

integration-tests/prometheus-java-client-init.patch

+--- a/prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/registry/MetricNameFilter.java
+++ b/prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/registry/MetricNameFilter.java
+@@ -8,6 +8,8 @@ import java.util.Collection;
+ import java.util.function.Predicate;
+
+ /** Filter samples (i.e. time series) by name. */
+// XXX: Don't suggest changes when there are local methods with the same name.
+@SuppressWarnings("ExplicitArgumentEnumeration")
+ public class MetricNameFilter implements Predicate<String> {
+
+   /** For convenience, a filter that allows all names. */
+--- a/prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/snapshots/Quantiles.java
+++ b/prometheus-metrics-model/src/main/java/io/prometheus/metrics/model/snapshots/Quantiles.java
+@@ -41,6 +41,8 @@ public class Quantiles implements Iterable<Quantile> {
+    * Create a new Quantiles instance. You can either create Quantiles with one of the static {@code
+    * Quantiles.of(...)} methods, or you can use the {@link Quantiles#builder()}.
+    */
+  // XXX: Don't suggest changes when there are local methods with the same name.
+  @SuppressWarnings("ExplicitArgumentEnumeration")
+   public static Quantiles of(Quantile... quantiles) {
+     return of(Arrays.asList(quantiles));
+   }


As discussed offline, we should try to fix ExplicitArgumentEnumeration so that thee suppressions aren't necessary. (I may have time for this ~tomorrow.)

I may have time for this ~tomorrow

If not, I'll try to make time for it on Monday! Today won't work for me.

I filed #1475 and rebased this PR to exclude these suppressions from the patch file.

Stephan202 · 2024-12-21T11:45:11Z

integration-tests/prometheus-java-client-init.patch

+--- a/prometheus-metrics-model/src/test/java/io/prometheus/metrics/model/snapshots/MetricSnapshotTest.java
+++ b/prometheus-metrics-model/src/test/java/io/prometheus/metrics/model/snapshots/MetricSnapshotTest.java
+@@ -7,6 +7,8 @@ import org.junit.jupiter.api.Test;
+
+ class MetricSnapshotTest {
+
+  // XXX: Investigate the `satisfies` and how to rewrite this.
+  @SuppressWarnings("AssertThatThrownBy")
+   @Test
+   public void testDuplicateLabels() {
+     assertThatExceptionOfType(DuplicateLabelsException.class)
+--- a/prometheus-metrics-model/src/test/java/io/prometheus/metrics/model/snapshots/PrometheusNamingTest.java
+++ b/prometheus-metrics-model/src/test/java/io/prometheus/metrics/model/snapshots/PrometheusNamingTest.java
+@@ -6,6 +6,8 @@ import static org.assertj.core.api.Assertions.assertThatExceptionOfType;
+
+ import org.junit.jupiter.api.Test;
+
+// XXX: Fix this edge case in the `JUnitMethodDeclaration` check.
+@SuppressWarnings("JUnitMethodDeclaration")
+ class PrometheusNamingTest {
+
+   @Test


We should either fix these cases or formulate better XXX comments. (Can have a look ~tomorrow.)

github-actions · 2024-12-21T12:18:01Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

rickie · 2024-12-22T12:41:03Z

Found a few flaky test issues on their GitHub about the same class as what you noticed: prometheus/client_java#956, prometheus/client_java#946.

I also experienced the issue on my local machine. Maybe for now we should report and exclude the specific case via -Dtest?

github-actions · 2024-12-22T12:41:39Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-22T13:25:02Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202 · 2024-12-22T19:41:48Z

I also experienced the issue on my local machine. Maybe for now we should report and exclude the specific case via -Dtest?

Yeah, agree. I filed prometheus/client_java#1242 and will reference that issue in an accompanying comment.

github-actions · 2024-12-22T21:19:08Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202 · 2024-12-22T21:20:27Z

Rebased and added a commit. The PR now requires #1475 and #1476 to be merged first. Open points (not today):

Cleanup of .github/workflows/run-integration-tests.yml.
A closer look at the generated changes.
A closer look at the @SuppressWarnings("AssertThatThrownBy") in the initial patch file.

And disable flaky test.

github-actions · 2024-12-23T16:58:55Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-23T17:25:48Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

Stephan202 · 2024-12-23T17:27:19Z

Added a commit to document some possible follow-up work. Open points:

Fixing the build. Somehow pulling Docker images keeps failing:

[main] INFO tc.testcontainers/ryuk:0.11.0 - Pulling docker image: testcontainers/ryuk:0.11.0. Please be patient; this may take some time but only needs to be done once.
[docker-java-stream--1295187651] ERROR com.github.dockerjava.api.async.ResultCallbackTemplate - Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Head \"[https://registry-1.docker.io/v2/testcontainers/ryuk/manifests/0.11.0\](https://registry-1.docker.io/v2/testcontainers/ryuk/manifests/0.11.0/)": Get \"https://auth.docker.io/token?account=githubactions&scope=repository%3Atestcontainers%2Fryuk%3Apull&service=registry.docker.io\": dial tcp 54.185.253.63:443: connect: connection refused"}

Cleanup of .github/workflows/run-integration-tests.yml

github-actions · 2024-12-23T17:45:07Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-23T18:16:15Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-23T21:26:22Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

github-actions · 2024-12-23T22:15:37Z

Looks good. No mutations were possible for these changes.
Mutation testing report by Pitest. Review any surviving mutants by inspecting the line comments under Files changed.

sonarqubecloud · 2024-12-23T22:15:54Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

rickie force-pushed the rossendrijver/prometheus_integration_tests branch from 3cfe7b9 to 90a8bdd Compare December 18, 2024 13:31

rickie added this to the 0.20.0 milestone Dec 19, 2024

rickie added the chore A task not related to code (build, formatting, process, ...) label Dec 19, 2024

rickie commented Dec 19, 2024

View reviewed changes

rickie force-pushed the rossendrijver/prometheus_integration_tests branch from 3421c71 to b4c5bed Compare December 20, 2024 13:43

rickie requested review from Stephan202 and mohamedsamehsalah December 20, 2024 13:44

rickie marked this pull request as ready for review December 20, 2024 13:45

mohamedsamehsalah approved these changes Dec 20, 2024

View reviewed changes

Stephan202 force-pushed the rossendrijver/prometheus_integration_tests branch from b4c5bed to 75534db Compare December 21, 2024 12:15

Stephan202 reviewed Dec 21, 2024

View reviewed changes

rickie force-pushed the rossendrijver/prometheus_integration_tests branch from 75534db to e8ff396 Compare December 22, 2024 12:39

This was referenced Dec 22, 2024

Improve ExplicitArgumentEnumeration check #1475

Merged

Fix flaky SlidingWindowTest prometheus/client_java#1242

Open

Stephan202 mentioned this pull request Dec 22, 2024

Improve JUnitMethodDeclaration method rename heuristics #1476

Merged

Stephan202 force-pushed the rossendrijver/prometheus_integration_tests branch from 61ccda6 to a8d83fb Compare December 22, 2024 21:16

rickie and others added 15 commits December 23, 2024 17:56

Simplify patch

6a03260

Update expected changes with one line number changed

17b3121

Simplify further

439548b

Revert: changes to test GHA

aaa6491

Use "correct" logger name

559ccaa

Add Guava upgrade pointer for Renovate

e077fa4

Make some changes to get it to work with ExplicitArgumentEnumaration

f4e615f

Drop unneccesary Guava changes from init-patch

e62867c

Make diff smaller

1627bd1

Update patch with current state

8b5c381

Suggestions

60d1ef6

Drop ExplicitArgumentEnumeration suppressions

5142e29

Drop JUnitMethodDeclaration suppression

9c4897a

And disable flaky test.

Allow Testcontainers to pull images from Docker Hub

4da4c7e

Drop AssertThatThrownBy suppression

758bf21

Stephan202 force-pushed the rossendrijver/prometheus_integration_tests branch from 77ddee5 to 758bf21 Compare December 23, 2024 16:56

Document possible follow-up work

5fc04fb

Worth a shot

5a44e2c

Even more...

8b4ee4a

Skip PushGatewayIT tests

231bfc6

Stephan202 added 2 commits December 23, 2024 23:12

Document findings

393f08a

Trigger all integration tests

423c150

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Introduce Prometheus Java Client integration test #1468

Introduce Prometheus Java Client integration test #1468

rickie commented Dec 18, 2024 •

edited

Loading

github-actions bot commented Dec 18, 2024

github-actions bot commented Dec 18, 2024

github-actions bot commented Dec 19, 2024

rickie commented Dec 19, 2024 •

edited

Loading

github-actions bot commented Dec 19, 2024

rickie Dec 19, 2024

github-actions bot commented Dec 20, 2024

github-actions bot commented Dec 20, 2024

Stephan202 left a comment

Stephan202 Dec 21, 2024

Stephan202 Dec 21, 2024

Stephan202 Dec 21, 2024

Stephan202 Dec 21, 2024

rickie Dec 22, 2024 •

edited

Loading

Stephan202 Dec 22, 2024

Stephan202 Dec 21, 2024

github-actions bot commented Dec 21, 2024

rickie commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

Stephan202 commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

Stephan202 commented Dec 22, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

Stephan202 commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

sonarqubecloud bot commented Dec 23, 2024


		class PrometheusInstrumentationScopeTest {

		+ // XXX: Introduce support for rewriting `assertThatThrownBy#havingRootCause()`.

Introduce Prometheus Java Client integration test #1468

Are you sure you want to change the base?

Introduce Prometheus Java Client integration test #1468

Conversation

rickie commented Dec 18, 2024 • edited Loading

github-actions bot commented Dec 18, 2024

github-actions bot commented Dec 18, 2024

github-actions bot commented Dec 19, 2024

rickie commented Dec 19, 2024 • edited Loading

github-actions bot commented Dec 19, 2024

rickie Dec 19, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 20, 2024

github-actions bot commented Dec 20, 2024

Stephan202 left a comment

Choose a reason for hiding this comment

Stephan202 Dec 21, 2024

Choose a reason for hiding this comment

Stephan202 Dec 21, 2024

Choose a reason for hiding this comment

Stephan202 Dec 21, 2024

Choose a reason for hiding this comment

Stephan202 Dec 21, 2024

Choose a reason for hiding this comment

rickie Dec 22, 2024 • edited Loading

Choose a reason for hiding this comment

Stephan202 Dec 22, 2024

Choose a reason for hiding this comment

Stephan202 Dec 21, 2024

Choose a reason for hiding this comment

github-actions bot commented Dec 21, 2024

rickie commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

Stephan202 commented Dec 22, 2024

github-actions bot commented Dec 22, 2024

Stephan202 commented Dec 22, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

Stephan202 commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

github-actions bot commented Dec 23, 2024

sonarqubecloud bot commented Dec 23, 2024

Quality Gate passed

rickie commented Dec 18, 2024 •

edited

Loading

rickie commented Dec 19, 2024 •

edited

Loading

rickie Dec 22, 2024 •

edited

Loading