From ac2ddb5e8a7bf529871a288501233f5874b8c298 Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Thu, 8 Feb 2024 11:07:31 -0600 Subject: [PATCH 1/2] [apm] Add docs on a known issue with APM Server + APM Java agent (#3591) * add known issue * add footnote to apm compatibility matrix * address feedback from @simitt * address feedback from @bmorelli25 * Update docs/en/observability/apm/known-issues.asciidoc Co-authored-by: Silvia Mitter --------- Co-authored-by: Silvia Mitter (cherry picked from commit c151d92890e6a2965cb15d297b52a3377734359a) --- .../apm/agent-server-compatibility.asciidoc | 2 +- .../observability/apm/known-issues.asciidoc | 58 +++++++++++++++---- 2 files changed, 47 insertions(+), 13 deletions(-) diff --git a/docs/en/observability/apm/agent-server-compatibility.asciidoc b/docs/en/observability/apm/agent-server-compatibility.asciidoc index 3aadd899e2..33f935bd06 100644 --- a/docs/en/observability/apm/agent-server-compatibility.asciidoc +++ b/docs/en/observability/apm/agent-server-compatibility.asciidoc @@ -24,7 +24,7 @@ The chart below outlines the compatibility between different versions of Elastic |`1.x` |≥ `8.12` // Java -.1+|**Java agent** +.1+|*Java agent*{empty}footnote:[Java agent < 1.43.0 not fully compatible with APM Server >= 8.11.0. Read more in <>.] |`1.x`|≥ `6.5` // .NET diff --git a/docs/en/observability/apm/known-issues.asciidoc b/docs/en/observability/apm/known-issues.asciidoc index d44c9dd280..eee3edd41c 100644 --- a/docs/en/observability/apm/known-issues.asciidoc +++ b/docs/en/observability/apm/known-issues.asciidoc @@ -2,6 +2,49 @@ = Known issues APM has the following known issues: +//// +TEMPLATE +Note: Add known issues for newer Elastic Stack +versions to the top of this page + +*Brief description* + +_Versions: XX.XX.XX, YY.YY.YY, ZZ.ZZ.ZZ_ + +Detailed description including: + +The conditions in which this issue occurs +The behavior of the issue +Why it happens +If applicable, exact error messages linked to this issue so users searching for the error message end up here +Link to fix +//// + +[[apm-empty-metricset-values]] +*Upgrading APM Server to 8.11+ might break event intake from older APM Java agents* + +_APM Server versions: >=8.11.0_ + +_Elastic APM Java agent versions: < 1.43.0_ + +// Describe the conditions in which this issue occurs +If you are using APM Server (> v8.11.0) and the Elastic APM Java agent (< v1.43.0), +// Describe the behavior of the issue +the agent may be sending empty histogram metricsets. + +// Describe why it happens +In previous APM Server versions some data validation was not properly applied, +leading the APM Server to accept empty histogram metricsets where it shouldn't. +This bug was fixed in the APM Server in 8.11.0. + +The APM Java agent (< v1.43.0) was sending this kind of invalid data under certain circumstances. +If you upgrade the APM Server to v8.11.0+ _without_ upgrading the APM Java agent version, +metricsets can be rejected by the APM Server and can result in additional error logs in the Java agent. + +// Include exact error messages linked to this issue +// so users searching for the error message end up here. + +// Link to fix +The fix is to upgrade the Elastic APM Java agent to a version >= 1.43.0. +Find details in https://github.com/elastic/apm-data/pull/157[elastic/apm-data#157]. + *traces-apm@custom ingest pipeline applied to certain data streams unintentionally* + _APM Server versions: 8.12.0_ + @@ -26,10 +69,10 @@ A fix was released in 8.12.1: https://github.com/elastic/kibana/pull/175448[elas *Ingesting new JVM metrics in 8.9 and 8.10 breaks upgrade to 8.11 and stops ingestion* + _APM Server versions: 8.11.0, 8.11.1_ + -_Elastic APM Java Agent versions: 1.39.0+_ +_Elastic APM Java agent versions: 1.39.0+_ // Describe the conditions in which this issue occurs -If you're using the Elastic APM Java Agent v1.39.0+ to send new JVM metrics to APM Server v8.9.x and v8.10.x, +If you're using the Elastic APM Java agent v1.39.0+ to send new JVM metrics to APM Server v8.9.x and v8.10.x, // Describe the behavior of the issue upgrading to 8.11.0 or 8.11.1 will silently fail and stop ingesting APM metrics. // Describe why it happens @@ -55,14 +98,5 @@ Failed installing package [apm] due to error: [ResponseError: mapper_parsing_exc mapper_parsing_exception: Field [jvm.memory.non_heap.pool.committed] attempted to shadow a time_series_metric] ---- -// Link to fix? +// Link to fix A fix was released in 8.11.2: https://github.com/elastic/kibana/pull/171712[elastic/kibana#171712]. - -// TEMPLATE - -//// -*Brief description* + -_Versions: XX.XX.XX, YY.YY.YY, ZZ.ZZ.ZZ_ - -Detailed description. -//// From 4a1b71d904fc9b8f5596d1889d902833416543ce Mon Sep 17 00:00:00 2001 From: Colleen McGinnis Date: Thu, 8 Feb 2024 17:30:07 -0600 Subject: [PATCH 2/2] reorder known issues --- .../observability/apm/known-issues.asciidoc | 42 +++++++++---------- 1 file changed, 21 insertions(+), 21 deletions(-) diff --git a/docs/en/observability/apm/known-issues.asciidoc b/docs/en/observability/apm/known-issues.asciidoc index eee3edd41c..7da2edf919 100644 --- a/docs/en/observability/apm/known-issues.asciidoc +++ b/docs/en/observability/apm/known-issues.asciidoc @@ -19,6 +19,27 @@ If applicable, exact error messages linked to this issue so users searching for Link to fix //// +*traces-apm@custom ingest pipeline applied to certain data streams unintentionally* + +_APM Server versions: 8.12.0_ + + +// Describe the conditions in which this issue occurs +If you're using the Elastic APM Server v8.12.0, +// Describe the behavior of the issue +the `traces-apm@custom` ingest pipeline is now additionally applied to data streams `traces-apm.sampled-*` +and `traces-apm.rum-*`, and applied twice for `traces-apm-*`. This bug impacts users with a non-empty `traces-apm@custom` ingest pipeline. + +If you rely on this unintended behavior in 8.12.0, please rename your pipeline to `traces-apm.integration@custom` to preserve this behavior in later versions. + +// Describe why it happens +// This happens because... + +// Include exact error messages linked to this issue +// so users searching for the error message end up here. + +// Link to fix? +A fix was released in 8.12.1: https://github.com/elastic/kibana/pull/175448[elastic/kibana#175448]. + + [[apm-empty-metricset-values]] *Upgrading APM Server to 8.11+ might break event intake from older APM Java agents* + _APM Server versions: >=8.11.0_ + @@ -46,27 +67,6 @@ The fix is to upgrade the Elastic APM Java agent to a version >= 1.43.0. Find details in https://github.com/elastic/apm-data/pull/157[elastic/apm-data#157]. -*traces-apm@custom ingest pipeline applied to certain data streams unintentionally* + -_APM Server versions: 8.12.0_ + - -// Describe the conditions in which this issue occurs -If you're using the Elastic APM Server v8.12.0, -// Describe the behavior of the issue -the `traces-apm@custom` ingest pipeline is now additionally applied to data streams `traces-apm.sampled-*` -and `traces-apm.rum-*`, and applied twice for `traces-apm-*`. This bug impacts users with a non-empty `traces-apm@custom` ingest pipeline. - -If you rely on this unintended behavior in 8.12.0, please rename your pipeline to `traces-apm.integration@custom` to preserve this behavior in later versions. - -// Describe why it happens -// This happens because... - -// Include exact error messages linked to this issue -// so users searching for the error message end up here. - -// Link to fix? -A fix was released in 8.12.1: https://github.com/elastic/kibana/pull/175448[elastic/kibana#175448]. - - *Ingesting new JVM metrics in 8.9 and 8.10 breaks upgrade to 8.11 and stops ingestion* + _APM Server versions: 8.11.0, 8.11.1_ + _Elastic APM Java agent versions: 1.39.0+_