Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[8.16](backport #4582) Update quickstarts for 8.16 #4591

Merged
merged 2 commits into from
Nov 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file not shown.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified docs/en/observability/images/quickstart-k8s-entry-point.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -47,17 +47,17 @@ You can use an AWS CLI command or upload the template to the AWS CloudFormation
* `FirehoseStreamNameForLogs`: Name for Amazon Data Firehose Stream for collecting CloudWatch logs. Default is `elastic-firehose-logs`.
====

IMPORTANT: Some AWS services need additional manual configuration to properly ingest logs and metrics. For more information, check the
IMPORTANT: Some AWS services need additional manual configuration to properly ingest logs and metrics. For more information, check the
link:https://www.elastic.co/docs/current/integrations/aws[AWS integration] documentation.

Data collection with AWS Firehose is supported on ESS deployments in AWS, Azure and GCP.
Data collection with AWS Firehose is supported on ESS deployments in AWS, Azure and GCP.

[discrete]
== Prerequisites

* A deployment using our hosted {ess} on {ess-trial}[{ecloud}]. The deployment includes an {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data.
* A user with the `superuser` {ref}/built-in-roles.html[built-in role] or the privileges required to onboard data.
+
+
[%collapsible]
.Expand to view required privileges
====
Expand All @@ -75,16 +75,16 @@ NOTE: The default CloudFormation stack is created in the AWS region selected for
The AWS Firehose receiver has the following limitations:

* It does not support AWS PrivateLink.
* It is not available for on-premise Elastic Stack deployments.
* The CloudFormation template detects and ingests logs and metrics within a single AWS region only.
* It is not available for on-premise Elastic Stack deployments.
* The CloudFormation template detects and ingests logs and metrics within a single AWS region only.

The following table shows the type of data ingested by the supported AWS services:

|===
| AWS Service | Data type
| AWS Service | Data type

| VPC Flow Logs |Logs
| API Gateway|Logs, Metrics
| VPC Flow Logs |Logs
| API Gateway|Logs, Metrics
| CloudTrail | Logs
| Network Firewall | Logs, Metrics
| Route53 | Logs
Expand Down Expand Up @@ -113,9 +113,9 @@ The following table shows the type of data ingested by the supported AWS service
[discrete]
== Collect your data

. In {kib}, go to **Observability** and click **Add Data**.
. In {kib}, go to the **Observability** UI and click **Add Data**.

. Select **Cloud**, **AWS**, and then select **AWS Firehose**.
. Under **What do you want to monitor?** select **Cloud**, **AWS**, and then select **AWS Firehose**.
+
[role="screenshot"]
image::images/quickstart-aws-firehose-entry-point.png[AWS Firehose entry point]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[[quickstart-monitor-hosts-with-elastic-agent]]
= Quickstart: Monitor hosts with {agent}

preview::[]

In this quickstart guide, you'll learn how to scan your host to detect and collect logs and metrics,
then navigate to dashboards to further analyze and explore your observability data.
You'll also learn how to get value out of your observability data.
Expand All @@ -15,7 +13,7 @@ The script also generates an {agent} configuration file that you can use with yo
[discrete]
== Prerequisites

* A deployment using our hosted {ess} on {ess-trial}[{ecloud}]. The deployment includes an {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data.
* An {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out our hosted {ess} on {ess-trial}[{ecloud}].
* A user with the `superuser` {ref}/built-in-roles.html[built-in role] or the privileges required to onboard data.
+
[%collapsible]
Expand All @@ -30,23 +28,21 @@ The script also generates an {agent} configuration file that you can use with yo
[discrete]
== Limitations

* The auto-detection script currently scans for metrics and logs from Apache, Docker, Nginx, and the host system.
It also scans for custom log files.
* The auto-detection script works on Linux and MacOS only. Support for the `lsof` command is also required if you want to detect custom log files.
* If you've installed Apache or Nginx in a non-standard location, you'll need to specify log file paths manually when you run the scan.
* Because Docker Desktop runs in a VM, its logs are not auto-detected.

[discrete]
== Collect your data

. Go to the **Observability** UI and click **Add Data**.
. Select **Collect and analyze logs**, and then select **Auto-detect logs and metrics**.
. Copy the command that's shown. For example:
. In {kib}, go to the **Observability** UI and click **Add Data**.
. Under **What do you want to monitor?** select **Host**, and then select **Elastic Agent: Logs & Metrics**.
+
[role="screenshot"]
image::images/quickstart-autodetection-command.png[Quick start showing command for running auto-detection]
image::images/quickstart-monitor-hosts-entry-point.png[Host monitoring entry point]
. Copy the install command.
+
You'll run this command to download the auto-detection script and scan your system for observability data.
You'll run this command to download the auto-detection script, scan your system for observability data, and install {agent}.
. Open a terminal on the host you want to scan, and run the command.
. Review the list of log files:
* Enter `Y` to ingest all the log files listed.
Expand All @@ -59,6 +55,7 @@ There might be a slight delay before logs and other data are ingested.
*****
**Need to scan your host again?**

The auto-detection script (`auto_detect.sh`) is downloaded to the directory where you ran the installation command.
You can re-run the script on the same host to detect additional logs.
The script will scan the host and reconfigure {agent} with any additional logs that are found.
If the script misses any custom logs, you can add them manually by entering `n` after the script has finished scanning the host.
Expand All @@ -75,23 +72,27 @@ the page may link to the following integration assets:
|====
| Integration asset | Description

| **System**
| Prebuilt dashboard for monitoring host status and health using system metrics.

| **Apache**
| Prebuilt dashboard for monitoring Apache HTTP server health using error and access log data.

| **Custom .log files**
| Logs Explorer for analyzing custom logs.

| **Docker**
| Prebuilt dashboard for monitoring the status and health of Docker containers.

| **MySQL**
| Prebuilt dashboard for monitoring MySQl server health using error and access log data.

| **Nginx**
| Prebuilt dashboard for monitoring Nginx server health using error and access log data.

| **System**
| Prebuilt dashboard for monitoring host status and health using system metrics.

| **Custom .log files**
| Logs Explorer for analyzing custom logs.
| **Other prebuilt dashboards**
| Prebuilt dashboards are also available for systems and services not described here,
including PostgreSQL, Redis, HAProxy, Kafka, RabbitMQ, Prometheus, Apache Tomcat, and MongoDB.
|====

For example, you can navigate the **Host overview** dashboard to explore detailed metrics about system usage and throughput.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
[[monitor-k8s-logs-metrics-with-elastic-agent]]
= Quickstart: Monitor your Kubernetes cluster with {agent}

preview::[]

In this quickstart guide, you'll learn how to create the Kubernetes resources that are required to monitor your cluster infrastructure.

This new approach requires minimal configuration and provides you with an easy setup to monitor your infrastructure. You no longer need to download, install, or configure the Elastic Agent, everything happens automatically when you run the kubectl command.
Expand All @@ -12,7 +10,7 @@ The kubectl command installs the standalone Elastic Agent in your Kubernetes clu
[discrete]
== Prerequisites

* A deployment using our hosted {ess} on {ess-trial}[{ecloud}]. The deployment includes an {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data.
* An {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out our hosted {ess} on {ess-trial}[{ecloud}].
* A user with the `superuser` {ref}/built-in-roles.html[built-in role] or the privileges required to onboard data.
+
[%collapsible]
Expand All @@ -28,9 +26,9 @@ The kubectl command installs the standalone Elastic Agent in your Kubernetes clu
[discrete]
== Collect your data

. Go to the **Observability** UI and click **Add Data**.
. In {kib}, go to the **Observability** UI and click **Add Data**.

. Select **Monitor infrastructure**, and then select **Kubernetes**.
. Under **What do you want to monitor?** select **Kubernetes**, and then select **Elastic Agent: Logs & Metrics**.
+
[role="screenshot"]
image::images/quickstart-k8s-entry-point.png[Kubernetes entry point]
Expand Down
10 changes: 5 additions & 5 deletions docs/en/observability/quickstarts/monitor-k8s-otel.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

preview::[]

In this quickstart guide, you will learn how to send Kubernetes logs, metrics, and application traces to Elasticsearch, using the https://github.com/open-telemetry/opentelemetry-operator/[OpenTelemetry Operator] to orchestrate https://github.com/elastic/opentelemetry/tree/main[Elastic Distributions of OpenTelemetry] (EDOT) Collectors and SDK instances.
In this quickstart guide, you'll learn how to send Kubernetes logs, metrics, and application traces to Elasticsearch, using the https://github.com/open-telemetry/opentelemetry-operator/[OpenTelemetry Operator] to orchestrate https://github.com/elastic/opentelemetry/tree/main[Elastic Distributions of OpenTelemetry] (EDOT) Collectors and SDK instances.

All the components will be deployed through the https://github.com/open-telemetry/opentelemetry-helm-charts/tree/main/charts/opentelemetry-kube-stack[opentelemetry-kube-stack] helm chart. They include:

Expand All @@ -17,7 +17,7 @@ For a more detailed description of the components and advanced configuration, re
[discrete]
== Prerequisites

* A deployment using our hosted {ess} on {ess-trial}[{ecloud}]. The deployment includes an {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data.
* An {es} cluster for storing and searching your data, and {kib} for visualizing and managing your data. This quickstart is available for all Elastic deployment models. To get started quickly, try out our hosted {ess} on {ess-trial}[{ecloud}].
* A running Kubernetes cluster (v1.23 or newer).
* https://kubernetes.io/docs/reference/kubectl/[Kubectl].
* https://helm.sh/docs/intro/install/[Helm].
Expand All @@ -26,9 +26,9 @@ For a more detailed description of the components and advanced configuration, re
[discrete]
== Collect your data

. In {kib}, go to the **Observability** UI and click **Add Data**.
. In {kib}, go to the **Observability** UI and click **Add Data**.

. Under *`What do you want to monitor?`*, select **Kubernetes**, and then select the **OpenTelemetry: Full Observability** option.
. Under **What do you want to monitor?** select **Kubernetes**, and then select **OpenTelemetry: Full Observability**.
+
[role="screenshot"]
image::images/quickstart-k8s-otel-entry-point.png[Kubernetes-OTel entry point]
Expand All @@ -41,7 +41,7 @@ The default installation deploys the OpenTelemetry Operator with a self-signed T
====
+
Deploy the OpenTelemetry Operator and EDOT Collectors using the kube-stack Helm chart with the provided `values.yaml` file. You will run a few commands to:
+
+
* Add the helm chart repository needed for the installation.
* Create a namespace.
* Create a secret with an API Key and the {es} endpoint to be used by the collectors.
Expand Down