Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[proof of concept] Restructure APM content related to APM agents and OpenTelemetry? #4096

Closed
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions docs/en/observability/apm.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -48,14 +48,14 @@ Want to host everything yourself instead? See <<apm-getting-started-apm-server>>

include::{observability-docs-root}/docs/en/observability/apm/getting-started-apm-server.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/collect-application-data.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/data-model.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/features.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/how-to.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/open-telemetry.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/manage-storage.asciidoc[]

include::{observability-docs-root}/docs/en/observability/apm/configure/index.asciidoc[leveloffset=+1]
Expand Down
16 changes: 10 additions & 6 deletions docs/en/observability/apm/api-keys.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ IMPORTANT: API keys are sent as plain-text,
so they only provide security when used in combination with <<apm-agent-tls,TLS>>.

When enabled, API keys are used to authorize requests to the APM Server.
API keys are not applicable for APM agents running on clients, like the RUM agent,
API keys are not applicable for _{generic_term_for_agent_distro}_ s running on clients, like the RUM agent,
as there is no way to prevent them from being publicly exposed.

You can assign one or more unique privileges to each API key:
Expand All @@ -14,13 +14,13 @@ You can assign one or more unique privileges to each API key:
{kibana-ref}/agent-configuration.html[Agent configuration remotely].
* *Ingest* (`event:write`): Required for ingesting agent events.

To secure the communication between APM Agents and the APM Server with API keys,
To secure the communication between _{generic_term_for_agent_distro}_ and the APM Server with API keys,
make sure <<apm-agent-tls,TLS>> is enabled, then complete these steps:

. <<apm-enable-api-key,Enable API keys>>
. <<apm-create-api-key-user,Create an API key user>>
. <<apm-create-an-api-key,Create an API key in {kib}>>
. <<apm-agent-api-key,Set the API key in your APM agents>>
. <<apm-agent-api-key,Set the API key in your _{generic_term_for_agent_distro}_>>

[[apm-enable-api-key]]
[float]
Expand Down Expand Up @@ -93,9 +93,9 @@ image::images/apm-ui-api-key.png[{apm-app} API key]

[[apm-agent-api-key]]
[float]
=== Set the API key in your APM agents
=== Set the API key in your _{generic_term_for_agent_distro}_

You can now apply your newly created API keys in the configuration of each of your APM agents.
You can now apply your newly created API keys in the configuration of each of your _{generic_term_for_agent_distro}_.
See the relevant agent documentation for additional information:

// Not relevant for RUM
Expand All @@ -109,6 +109,10 @@ See the relevant agent documentation for additional information:
* *Python agent*: {apm-py-ref}/configuration.html#config-api-key[`api_key`]
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-api-key[`api_key`]

* Java distro
* .NET distro
* Node distro

[[apm-configure-api-key-alternative]]
[float]
=== Alternate API key creation methods
Expand Down Expand Up @@ -252,7 +256,7 @@ The response will look similar to this:
}
----

The `credential` string, which is what agents use to communicate with APM Server,
The `credential` string, which is what _{generic_term_for_agent_distro}_ use to communicate with APM Server,
is a base64 encoded representation of the API key's `id:api_key`.
It can be created like this:

Expand Down
55 changes: 55 additions & 0 deletions docs/en/observability/apm/collect-application-data.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
[[apm-collect-application-data]]
== Collect application data

:generic_term_for_agent_distro: <generic term for APM agent *or* Otel distro>

There are several ways you can collect application data to send to Elastic.
Which option is best for your use case depends on several factors including:
<something>, <something>, and <something>.

////
Compare/contrast methods for collection application data
////
[discrete]
=== Elastic Distributions for OpenTelemetry

// What is it
<What is it>

// Why use it
<Why would you choose this option>

// Where to learn more
Read more about how to use Elastic APM agents in <<apm-elastic-distributions-for-open-telemetry>>.

[discrete]
=== Elastic APM agents

// What is it
<What is it>

// Why use it
<Why would you choose this option>

// Where to learn more
Read more about how to use Elastic APM agents in <<apm-agents>>.

[discrete]
=== Elastic OpenTelemetry integration

// What is it
<What is it>

// Why use it
<Why would you choose this option>

// Where to learn more
Read more about how to use Elastic APM agents in <<apm-open-telemetry>>.

include::secure-agent-communication.asciidoc[leveloffset=+1]

include::elastic-distributions-for-open-telemetry.asciidoc[leveloffset=+1]

include::elastic-apm-agents.asciidoc[leveloffset=+1]

include::open-telemetry.asciidoc[leveloffset=+1]
4 changes: 4 additions & 0 deletions docs/en/observability/apm/elastic-apm-agents.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[[apm-agents]]
== Elastic APM agents

<Add info that is relevant to _all_ APM agents here>
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[[apm-elastic-distributions-for-open-telemetry]]
== Elastic Distributions for OpenTelemetry

<Add info that is relevant to _all_ Elastic Distros for Open Telemetry here>
39 changes: 17 additions & 22 deletions docs/en/observability/apm/getting-started-apm-server.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ See <<traces-get-started,getting started documentation>> to get started in minut
IMPORTANT: Starting in version 8.0.0, {fleet} uses the APM integration to set up and manage APM index templates,
{ilm-init} policies, and ingest pipelines. APM Server will only send data to {es} _after_ the APM integration has been installed.

The APM Server receives performance data from your APM agents,
The APM Server receives performance data from your _{generic_term_for_agent_distro}_,
validates and processes it, and then transforms the data into {es} documents.
If you're on this page, then you've chosen to self-manage the Elastic Stack,
and you now must decide how to run and configure the APM Server.
Expand Down Expand Up @@ -51,7 +51,7 @@ image::./images/bin-ov.png[APM Server binary overview]

**Required components**:

- APM agents
- APM agents or Elastic Distributions for OpenTelemetry
- APM Server
- {stack}

Expand Down Expand Up @@ -82,7 +82,7 @@ image::./images/fm-ov.png[APM Server fleet overview]

**Required components**:

- APM agents
- APM agents or Elastic Distributions for OpenTelemetry
- APM Server
- {agent}
- Fleet Server
Expand Down Expand Up @@ -451,20 +451,10 @@ See the <<apm-directory-layout,deb & rpm default paths>> for a full directory la
==== Step 4: Next steps

// Use a tagged region to pull APM Agent information from the APM Overview
If you haven't already, you can now install APM Agents in your services!

* {apm-android-ref}/intro.html[Android agent]
* {apm-go-ref-v}/introduction.html[Go agent]
* {apm-ios-ref-v}/intro.html[iOS agent]
* {apm-java-ref-v}/intro.html[Java agent]
* {apm-dotnet-ref-v}/intro.html[.NET agent]
* {apm-node-ref-v}/intro.html[Node.js agent]
* {apm-php-ref-v}/intro.html[PHP agent]
* {apm-py-ref-v}/getting-started.html[Python agent]
* {apm-ruby-ref-v}/introduction.html[Ruby agent]
* {apm-rum-ref-v}/intro.html[JavaScript Real User Monitoring (RUM) agent]

Once you have at least one {apm-agent} sending data to APM Server,
If you haven't already, you can now install _{generic_term_for_agent_distro}_ in your services!
To choose the right option for your use case, refer to <<apm-collect-application-data>>.

Once you have at least one _{generic_term_for_agent_distro}_ sending data to APM Server,
you can start visualizing your data in the {kibana-ref}/xpack-apm.html[{apm-app}].

If you're migrating from Jaeger, see <<apm-jaeger-integration>>.
Expand Down Expand Up @@ -510,13 +500,16 @@ include::{observability-docs-root}/docs/en/observability/apm/getting-started-apm
--
****

// Not just APM agents tho...
==== Step 3: Install APM agents

APM agents are written in the same language as your service.
To monitor a new service, you must install the agent and configure it with a service name,
_{generic_term_for_agent_distro}_ are written in the same language as your service.
To choose the right option for your use case, refer to <<apm-collect-application-data>>.

To monitor a new service, you must install the _{generic_term_for_agent_distro}_ and configure it with a service name,
APM Server host, and Secret token.

* **Service name**: The APM integration maps an instrumented service's name–defined in each {apm-agent}'s configuration–
* **Service name**: The APM integration maps an instrumented service's name–defined in each _{generic_term_for_agent_distro}_ 's configuration–
to the index that its data is stored in {es}.
Service names are case-insensitive and must be unique.
For example, you cannot have a service named `Foo` and another named `foo`.
Expand All @@ -525,11 +518,13 @@ Special characters will be removed from service names and replaced with undersco
* **APM Server URL**: The host and port that APM Server listens for events on.
This should match the host and port defined when setting up the APM integration.

* **Secret token**: Authentication method for {apm-agent} and APM Server communication.
* **Secret token**: Authentication method for _{generic_term_for_agent_distro}_ and APM Server communication.
This should match the secret token defined when setting up the APM integration.

TIP: You can edit your APM integration settings if you need to change the APM Server URL
or secret token to match your APM agents.
or secret token to match your _{generic_term_for_agent_distro}_.

<Add instructions for using Otel distros, too?>

include::{observability-docs-root}/docs/en/observability/apm/tab-widgets/install-agents-widget.asciidoc[]

Expand Down
2 changes: 2 additions & 0 deletions docs/en/observability/apm/open-telemetry.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
[[apm-open-telemetry]]
== OpenTelemetry integration

<Explain the difference between distros and this...>

https://opentelemetry.io/docs/concepts/what-is-opentelemetry/[OpenTelemetry] is a set of APIs, SDKs, tooling, and integrations that enable the capture and management of
telemetry data from your services and applications. For more information about the
OpenTelemetry project, see the https://github.com/open-telemetry/opentelemetry-specification/blob/master/README.md[spec].
Expand Down
10 changes: 9 additions & 1 deletion docs/en/observability/apm/secret-token.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,10 @@ Each Elastic {apm-agent} has a configuration option to set the value of the secr
* *Python agent*: {apm-py-ref}/configuration.html#config-secret-token[`secret_token`]
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-secret-token[`secret_token`]

* Java distro
* .NET distro
* Node distro

In addition to setting the secret token, ensure the configured server URL uses `HTTPS` instead of `HTTP`:

* *Go agent*: {apm-go-ref}/configuration.html#config-server-url[`ELASTIC_APM_SERVER_URL`]
Expand All @@ -50,4 +54,8 @@ In addition to setting the secret token, ensure the configured server URL uses `
* *Node.js agent*: {apm-node-ref}/configuration.html#server-url[`serverUrl`]
* *PHP agent*: {apm-php-ref-v}/configuration-reference.html#config-server-url[`server_url`]
* *Python agent*: {apm-py-ref}/[`server_url`]
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-server-url[`server_url`]
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-server-url[`server_url`]

* Java distro
* .NET distro
* Node distro
13 changes: 6 additions & 7 deletions docs/en/observability/apm/secure-agent-communication.asciidoc
Original file line number Diff line number Diff line change
@@ -1,22 +1,21 @@
[[apm-secure-agent-communication]]
== Secure communication with APM agents
== Secure communication with _{generic_term_for_agent_distro}_

++++
<titleabbrev>With APM agents</titleabbrev>
<titleabbrev>Secure communication</titleabbrev>
++++

Communication between APM agents and {agent} can be both encrypted and authenticated.
Communication between _{generic_term_for_agent_distro}_ and {agent} can be both encrypted and authenticated.
It is strongly recommended to use both TLS encryption and authentication as secrets are sent as plain text.

* <<apm-agent-tls,TLS encryption>>
* <<apm-api-key,API key authentication>>
* <<apm-secret-token,Secret token authentication>>

As soon as an authenticated communication is enabled,
requests without a valid token or API key will be denied.
If both API keys and a secret token are enabled, APM agents can choose whichever mechanism they support.
As soon as an authenticated communication is enabled, requests without a valid token or API key will be denied.
If both API keys and a secret token are enabled, _{generic_term_for_agent_distro}_ can choose whichever mechanism they support.

In some use-cases, like when an {apm-agent} is running on the client side,
In some use cases, like when an _{generic_term_for_agent_distro}_ is running on the client side,
authentication is not possible. See <<apm-anonymous-auth>> for more information.

include::./tls-comms.asciidoc[]
Expand Down
2 changes: 1 addition & 1 deletion docs/en/observability/apm/secure-comms.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ process and connecting securely to APM agents and the {stack}.
* <<apm-secure-comms-stack,Secure communication with the {stack}>>

:leveloffset: +1
include::secure-agent-communication.asciidoc[]
// include::secure-agent-communication.asciidoc[]

// APM privileges
include::{observability-docs-root}/docs/en/observability/apm/feature-roles.asciidoc[]
Expand Down
18 changes: 13 additions & 5 deletions docs/en/observability/apm/tls-comms.asciidoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
[[apm-agent-tls]]
=== {apm-agent} TLS communication
=== TLS communication

TLS is disabled by default.
When TLS is enabled for APM Server inbound communication, agents will verify the identity
When TLS is enabled for APM Server inbound communication, _{generic_term_for_agent_distro}_ will verify the identity
of the APM Server by authenticating its certificate.

When TLS is enabled, a certificate and corresponding private key are required.
Expand Down Expand Up @@ -36,10 +36,10 @@ include::{observability-docs-root}/docs/en/observability/apm/tab-widgets/tls-wid

[float]
[[apm-agent-self-sign-3]]
==== Step 3: Configure APM agents
==== Step 3: Configure _{generic_term_for_agent_distro}_

When the APM server uses a certificate that is not chained to a publicly-trusted certificate
(e.g. self-signed), additional configuration is required in the {apm-agent}:
(e.g. self-signed), additional configuration is required in the _{generic_term_for_agent_distro}_:

* *Go agent*: certificate pinning through {apm-go-ref}/configuration.html#config-server-cert[`ELASTIC_APM_SERVER_CERT`]
* *Python agent*: certificate pinning through {apm-py-ref}/configuration.html#config-server-cert[`server_cert`]
Expand All @@ -49,6 +49,10 @@ When the APM server uses a certificate that is not chained to a publicly-trusted
* *Java agent*: adding the certificate to the JVM `trustStore`.
See {apm-java-ref}/ssl-configuration.html#ssl-server-authentication[APM Server authentication] for more details.

* Java distro
* .NET distro
* Node distro

We do not recommend disabling {apm-agent} verification of the server's certificate, but it is possible:

* *Go agent*: {apm-go-ref}/configuration.html#config-verify-server-cert[`ELASTIC_APM_VERIFY_SERVER_CERT`]
Expand All @@ -59,9 +63,13 @@ We do not recommend disabling {apm-agent} verification of the server's certifica
* *Ruby agent*: {apm-ruby-ref}/configuration.html#config-verify-server-cert[`verify_server_cert`]
* *Node.js agent*: {apm-node-ref}/configuration.html#validate-server-cert[`verifyServerCert`]

* Java distro
* .NET distro
* Node distro

[float]
[[apm-agent-client-cert]]
=== Client certificate authentication

APM Server does not require agents to provide a certificate for authentication,
APM Server does not require _{generic_term_for_agent_distro}_ s to provide a certificate for authentication,
and there is no dedicated support for SSL/TLS client certificate authentication in Elastic’s backend agents.