docs: add Alloy to open source GreptimeDB (#1413)

docs/user-guide/ingest-data/for-observerbility/alloy.md

@@ -0,0 +1,120 @@
+---
+keywords: [Grafana Alloy, Prometheus Remote Write, OpenTelemetry, data pipeline]
+description: Instructions on integrating GreptimeDB with Grafana Alloy for Prometheus Remote Write and OpenTelemetry.
+---
+
+# Grafana Alloy
+
+[Grafana Alloy](https://grafana.com/docs/alloy/latest/) is an observability data pipeline for OpenTelemetry (OTel), Prometheus, Pyroscope, Loki, and many other metrics, logs, traces, and profiling tools.
+You can integrate GreptimeDB as a data sink for Alloy.
+
+## Prometheus Remote Write
+
+Configure GreptimeDB as remote write target:
+
+```hcl
+prometheus.remote_write "greptimedb" {
+    endpoint {
+        url = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/prometheus/write?db=${GREPTIME_DB:=public}"
+
+        basic_auth {
+            username = "${GREPTIME_USERNAME}"
+            password = "${GREPTIME_PASSWORD}"
+        }
+    }
+}
+```
+
+- `GREPTIME_HOST`: GreptimeDB host address, e.g., `localhost`.
+- `GREPTIME_DB`: GreptimeDB database name, default is `public`.
+- `GREPTIME_USERNAME` and `GREPTIME_PASSWORD`: The [authentication credentials](/user-guide/deployments/authentication/static.md) for GreptimeDB.
+
+For details on the data model transformation from Prometheus to GreptimeDB, refer to the [Data Model](/user-guide/ingest-data/for-observerbility/prometheus.md#data-model) section in the Prometheus Remote Write guide.
+
+## OpenTelemetry
+
+GreptimeDB can also be configured as OpenTelemetry collector.
+
+### Metrics
+
+```hcl
+otelcol.exporter.otlphttp "greptimedb" {
+  client {
+    endpoint = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/otlp/"
+    headers  = {
+      "X-Greptime-DB-Name" = "${GREPTIME_DB:=public}",
+    }
+    auth     = otelcol.auth.basic.credentials.handler
+  }
+}
+
+otelcol.auth.basic "credentials" {
+  username = "${GREPTIME_USERNAME}"
+  password = "${GREPTIME_PASSWORD}"
+}
+```
+
+- `GREPTIME_HOST`: GreptimeDB host address, e.g., `localhost`.
+- `GREPTIME_DB`: GreptimeDB database name, default is `public`.
+- `GREPTIME_USERNAME` and `GREPTIME_PASSWORD`: The [authentication credentials](/user-guide/deployments/authentication/static.md) for GreptimeDB.
+
+For details on the metrics data model transformation from OpenTelemetry to GreptimeDB, refer to the [Data Model](/user-guide/ingest-data/for-observerbility/opentelemetry.md#data-model) section in the OpenTelemetry guide.
+
+### Logs
+
+The following example setting up a logging pipeline using Loki and OpenTelemetry Collector (otelcol) to forward logs to a GreptimeDB:
+
+```hcl
+loki.source.file "greptime" {
+  targets = [
+    {__path__ = "/tmp/foo.txt"},
+  ]
+  forward_to = [otelcol.receiver.loki.greptime.receiver]
+}
+
+otelcol.receiver.loki "greptime" {
+  output {
+    logs = [otelcol.exporter.otlphttp.greptimedb_logs.input]
+  }
+}
+
+otelcol.auth.basic "credentials" {
+  username = "${GREPTIME_USERNAME}"
+  password = "${GREPTIME_PASSWORD}"
+}
+
+otelcol.exporter.otlphttp "greptimedb_logs" {
+  client {
+    endpoint = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/otlp/"
+    headers  = {
+      "X-Greptime-DB-Name" = "${GREPTIME_DB:=public}",
+      "X-Greptime-Log-Table-Name" = "demo_logs",
+      "X-Greptime-Gog-Extract-Keys" = "filename,log.file.name,loki.attribute.labels",
+    }
+    auth     = otelcol.auth.basic.credentials.handler
+  }
+}
+```
+
+- Loki Source Configuration
+  - The `loki.source.file "greptime"` block defines a source for Loki to read logs from a file located at `/tmp/foo.txt`
+  - The `forward_to` array indicates that the logs read from this file should be forwarded to the `otelcol.receiver.loki.greptime.receiver`
+- OpenTelemetry Collector Receiver Configuration:
+  - The `otelcol.receiver.loki "greptime"` block sets up a receiver within the OpenTelemetry Collector to receive logs from Loki.
+  - The `output` section specifies that the received logs should be forwarded to the `otelcol.exporter.otlphttp.greptimedb_logs.input`.
+- OpenTelemetry Collector Exporter Configuration:
+  - The `otelcol.exporter.otlphttp "greptimedb_logs"` block configures an HTTP exporter to send logs to GreptimeDB.
+  - `GREPTIME_HOST`: GreptimeDB host address, e.g., `localhost`.
+  - `GREPTIME_DB`: GreptimeDB database name, default is `public`.
+  - `GREPTIME_USERNAME` and `GREPTIME_PASSWORD`: The [authentication credentials](/user-guide/deployments/authentication/static.md) for GreptimeDB.
+  - `LOG_TABLE_NAME`: The name of the table to store logs, default table name is `opentelemetry_logs`.
+  - `EXTRACT_KEYS`: The keys to extract from the attributes, separated by commas (`,`), e.g., `filename,log.file.name,loki.attribute.labels`, see [HTTP API documentation](opentelemetry.md#otlphttp-api-1) for details.
+
+For details on the log data model transformation from OpenTelemetry to GreptimeDB, refer to the [Data Model](/user-guide/ingest-data/for-observerbility/opentelemetry.md#data-model-1) section in the OpenTelemetry guide.
+
+:::tip NOTE
+The example codes above may be outdated according to OpenTelemetry. We recommend that you refer to the official OpenTelemetry documentation And Grafana Alloy for the most up-to-date information.
+:::
+
+For more information on the example code, please refer to the official documentation for your preferred programming language.
+

docs/user-guide/ingest-data/for-observerbility/opentelemetry.md

@@ -7,15 +7,33 @@ description: Instructions for integrating OpenTelemetry with GreptimeDB, includi
 
 [OpenTelemetry](https://opentelemetry.io/) is a vendor-neutral open-source observability framework for instrumenting, generating, collecting, and exporting telemetry data such as traces, metrics, logs. The OpenTelemetry Protocol (OTLP) defines the encoding, transport, and delivery mechanism of telemetry data between telemetry sources, intermediate processes such as collectors and telemetry backends.
 
+## OpenTelemetry Collectors
+
+You can easily configure GreptimeDB as the target for your OpenTelemetry collector.
+For more information, please refer to the [Grafana Alloy](alloy.md) example.
+
 ## Metrics
 
-### OTLP/HTTP
+GreptimeDB is an observability backend to consume OpenTelemetry Metrics natively via [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol.
+
+### OTLP/HTTP API
+
+To send OpenTelemetry Metrics to GreptimeDB through OpenTelemetry SDK libraries, use the following information:
 
-import Includeotlpmetrycsintegration from '../../../db-cloud-shared/clients/otlp-metrics-integration.md' 
+* URL: `http{s}://<host>/v1/otlp/v1/metrics`
+* Headers:
+  * `X-Greptime-DB-Name`: `<dbname>`
+  * `Authorization`: `Basic` authentication, which is a Base64 encoded string of `<username>:<password>`. For more information, please refer to [Authentication](https://docs.greptime.com/user-guide/deployments/authentication/static/) and [HTTP API](https://docs.greptime.com/user-guide/protocols/http#authentication)
+
+The request uses binary protobuf to encode the payload, so you need to use packages that support `HTTP/protobuf`. For example, in Node.js, you can use [`exporter-trace-otlp-proto`](https://www.npmjs.com/package/@opentelemetry/exporter-trace-otlp-proto); in Go, you can use [`go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp`](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp); in Java, you can use [`io.opentelemetry:opentelemetry-exporter-otlp`](https://mvnrepository.com/artifact/io.opentelemetry/opentelemetry-exporter-otlp); and in Python, you can use [`opentelemetry-exporter-otlp-proto-http`](https://pypi.org/project/opentelemetry-exporter-otlp-proto-http/).
+
+:::tip NOTE
+The package names may change according to OpenTelemetry, so we recommend that you refer to the official OpenTelemetry documentation for the most up-to-date information.
+:::
 
-<Includeotlpmetrycsintegration/>
+For more information about the OpenTelemetry SDK, please refer to the official documentation for your preferred programming language.
 
-#### Example Code
+### Example Code
 
 Here are some example codes about how to setup the request in different languages:
 
@@ -95,7 +113,7 @@ The example codes above may be outdated according to OpenTelemetry. We recommend
 
 For more information on the example code, please refer to the official documentation for your preferred programming language.
 
-#### Data Model
+### Data Model
 
 The OTLP metrics data model is mapped to the GreptimeDB data model according to the following rules:
 
@@ -108,57 +126,34 @@ The OTLP metrics data model is mapped to the GreptimeDB data model according to
 
 ## Logs
 
-### OTLP/HTTP
-
-import Includeotlplogintegration from '../../../db-cloud-shared/clients/otlp-logs-integration.md' 
-
-<Includeotlplogintegration/>
-
-#### Example Code
-
-Here are some example codes about how to use Grafana Alloy to send OpenTelemetry logs to GreptimeDB:
-
-```hcl
-loki.source.file "greptime" {
-  targets = [
-    {__path__ = "/tmp/foo.txt"},
-  ]
-  forward_to = [otelcol.receiver.loki.greptime.receiver]
-}
-
-otelcol.receiver.loki "greptime" {
-  output {
-    logs = [otelcol.exporter.otlphttp.greptimedb_logs.input]
-  }
-}
-
-otelcol.auth.basic "credentials" {
-  username = "${GREPTIME_USERNAME}"
-  password = "${GREPTIME_PASSWORD}"
-}
-
-otelcol.exporter.otlphttp "greptimedb_logs" {
-  client {
-    endpoint = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/otlp/"
-    headers  = {
-      "X-Greptime-DB-Name" = "${GREPTIME_DB:=public}",
-      "X-Greptime-Log-Table-Name" = "demo_logs",
-      "X-Greptime-Gog-Extract-Keys" = "filename,log.file.name,loki.attribute.labels",
-    }
-    auth     = otelcol.auth.basic.credentials.handler
-  }
-}
-```
+GreptimeDB consumes OpenTelemetry Logs natively via [OTLP/HTTP](https://opentelemetry.io/docs/specs/otlp/#otlphttp) protocol.
+
+### OTLP/HTTP API API
+
+To send OpenTelemetry Logs to GreptimeDB through OpenTelemetry SDK libraries, use the following information:
+
+* URL: `http{s}://<host>/v1/otlp/v1/logs`
+* Headers:
+  * `X-Greptime-DB-Name`: `<dbname>`
+  * `Authorization`: `Basic` authentication, which is a Base64 encoded string of `<username>:<password>`. For more information, please refer to [Authentication](/user-guide/deployments/authentication/static.md) and [HTTP API](/user-guide/protocols/http.md#authentication).
+  * `X-Greptime-Log-Table-Name`: `<table_name>` (optional) - The table name to store the logs. If not provided, the default table name is `opentelemetry_logs`.
+  * `X-Greptime-Log-Extract-Keys`: `<extract_keys>` (optional) - The keys to extract from the attributes. The keys should be separated by commas (`,`). For example, `key1,key2,key3` will extract the keys `key1`, `key2`, and `key3` from the attributes and promote them to the top level of the log, setting them as tags. If the field type is array, float, or object, an error will be returned. If a pipeline is provided, this setting will be ignored.
+  * `X-Greptime-Log-Pipeline-Name`: `<pipeline_name>` (optional) - The pipeline name to process the logs. If not provided, the extract keys will be used to process the logs.
+  * `X-Greptime-Log-Pipeline-Version`: `<pipeline_version>` (optional) - The pipeline version to process the logs. If not provided, the latest version of the pipeline will be used.
 
-This example listens for changes to the file and sends the latest values to GreptimeDB via the otlp protocol.
+The request uses binary protobuf to encode the payload, so you need to use packages that support `HTTP/protobuf`.
 
 :::tip NOTE
-The example codes above may be outdated according to OpenTelemetry. We recommend that you refer to the official OpenTelemetry documentation And Grafana Alloy for the most up-to-date information.
+The package names may change according to OpenTelemetry, so we recommend that you refer to the official OpenTelemetry documentation for the most up-to-date information.
 :::
 
-For more information on the example code, please refer to the official documentation for your preferred programming language.
+For more information about the OpenTelemetry SDK, please refer to the official documentation for your preferred programming language.
+
+### Example Code
 
-#### Data Model
+Please refer to the [Alloy documentation](alloy.md#logs) for example code on how to send OpenTelemetry logs to GreptimeDB.
+
+### Data Model
 
 The OTLP logs data model is mapped to the GreptimeDB data model according to the following rules:
 
@@ -188,4 +183,5 @@ Default table schema:
 
 - You can use `X-Greptime-Log-Table-Name` to specify the table name for storing the logs. If not provided, the default table name is `opentelemetry_logs`.
 - All attributes, including resource attributes, scope attributes, and log attributes, will be stored as a JSON column in the GreptimeDB table.
-- The timestamp of the log will be used as the timestamp index in GreptimeDB, with the column name `timestamp`. It is preferred to use `time_unix_nano` as the timestamp column. If `time_unix_nano` is not provided, `observed_time_unix_nano` will be used instead.
+- The timestamp of the log will be used as the timestamp index in GreptimeDB, with the column name `timestamp`. It is preferred to use `time_unix_nano` as the timestamp column. If `time_unix_nano` is not provided, `observed_time_unix_nano` will be used instead.
+

docs/user-guide/ingest-data/for-observerbility/prometheus.md

@@ -10,6 +10,8 @@ providing a seamless integration experience.
 
 ## Remote write configuration
 
+### Prometheus configuration file
+
 To configure Prometheus with GreptimeDB,
 update your [Prometheus configuration file](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#configuration-file) (`prometheus.yml`) as follows:
 
@@ -33,6 +35,10 @@ remote_read:
 - The `db` parameter in the URL represents the database to which we want to write data. It is optional. By default, the database is set to `public`.
 - `basic_auth` is the authentication configuration. Fill in the username and password if GreptimeDB authentication is enabled. Please refer to the [authentication document](/user-guide/deployments/authentication/overview.md).
 
+### Grafana Alloy configuration file
+
+If you are using Grafana Alloy, configure the remote write endpoint in the Alloy configuration file (`config.alloy`). For more information, refer to the [Alloy documentation](alloy.md#prometheus-remote-write).
+
 ## Data Model
 
 In the [data model](/user-guide/concepts/data-model.md) of GreptimeDB, data is organized into tables with columns for tags, time index, and fields.

docs/user-guide/integrations/alloy.md

@@ -0,0 +1,10 @@
+---
+keywords: [Alloy, Grafana Alloy, GreptimeDB, Ingest Data]
+description: Integrate GreptimeDB with Grafana Alloy.
+---
+
+# Grafana Alloy
+
+GreptimeDB can be set up as a data sink for Grafana Alloy.
+For more information, please refer to the [Ingest Data through Grafana Alloy](/user-guide/ingest-data/for-observerbility/alloy.md) guide.
+