chore: add loki docs

GreptimeTeam · Nov 27, 2024 · 6c1359a · 6c1359a
1 parent a75d48d
commit 6c1359a
Show file tree

Hide file tree

Showing 4 changed files with 369 additions and 1 deletion.
diff --git a/docs/user-guide/ingest-data/for-observerbility/loki.md b/docs/user-guide/ingest-data/for-observerbility/loki.md
@@ -80,7 +80,8 @@ DESC loki_demo_logs;
 - line: The log message.
 
 if you specify the external labels, we will add them as tags to the table schema. like `job` and `from` in the above example.
-We can't specify tags manually, all labels are treated as tags and string type.
+You can't specify tags manually, all labels are treated as tags and string type.
+Please do not attempt to pre-create the table using SQL to specify tag columns, as this will cause a type mismatch and write failure.
 
 ### Example
 

diff --git a/...s-plugin-content-docs/current/user-guide/ingest-data/for-observerbility/loki.md b/...s-plugin-content-docs/current/user-guide/ingest-data/for-observerbility/loki.md
@@ -0,0 +1,122 @@
+# Loki
+
+## 使用方法
+
+### API
+
+要通过原始 HTTP API 将日志发送到 GreptimeDB，请使用以下信息：
+
+* URL: `http{s}://<host>/v1/loki/api/v1/push`
+* Headers:
+  * `X-Greptime-DB-Name`: `<dbname>`
+  * `Authorization`: `Basic` 认证，这是一个 Base64 编码的 `<username>:<password>` 字符串。更多信息，请参考 [认证](https://docs.greptime.com/user-guide/deployments/authentication/static/) 和 [HTTP API](https://docs.greptime.com/user-guide/protocols/http#authentication)。
+  * `X-Greptime-Log-Table-Name`: `<table_name>`（可选）- 存储日志的表名。如果未提供，默认表名为 `loki_logs`。
+
+请求使用二进制 protobuf 编码负载，定义的格式与 [logproto.proto](https://github.com/grafana/loki/blob/main/pkg/logproto/logproto.proto) 相同。
+
+### 示例代码
+
+[Grafana Alloy](https://grafana.com/docs/alloy/latest/) 是一个供应商中立的 OpenTelemetry (OTel) Collector 发行版。Alloy 独特地结合了社区中最好的开源可观测性信号。
+
+它提供了一个 Loki 导出器，可以用来将日志发送到 GreptimeDB。以下是一个配置示例：
+
+```hcl
+loki.source.file "greptime" {
+  targets = [
+    {__path__ = "/tmp/foo.txt"},
+  ]
+  forward_to = [loki.write.greptime_loki.receiver]
+}
+
+loki.write "greptime_loki" {
+    endpoint {
+        url = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/loki/api/v1/push"
+        headers  = {
+          "x-greptime-db-name" = "${GREPTIME_DB:=public}",
+          "x-greptime-log-table-name" = "${GREPTIME_LOG_TABLE_NAME:=loki_demo_logs}",
+        }
+    }
+    external_labels = {
+        "job" = "greptime"
+        "from" = "alloy"
+    }
+}
+```
+
+我们监听文件 `/tmp/foo.txt` 并将日志发送到 GreptimeDB。日志存储在表 `loki_demo_logs` 中，并带有 label `job` 和 `from`。
+
+更多信息，请参考 [Grafana Alloy loki.write 文档](https://grafana.com/docs/alloy/latest/reference/components/loki/loki.write/)。
+
+您可以运行以下命令来检查表中的数据：
+
+```sql
+SELECT * FROM loki_demo_logs;
++----------------------------+------------------------+--------------+-------+----------+
+| greptime_timestamp         | line                   | filename     | from  | job      |
++----------------------------+------------------------+--------------+-------+----------+
+| 2024-11-25 11:02:31.256251 | Greptime is very cool! | /tmp/foo.txt | alloy | greptime |
++----------------------------+------------------------+--------------+-------+----------+
+1 row in set (0.01 sec)
+```
+
+## 数据模型
+
+Loki 日志数据模型根据以下规则映射到 GreptimeDB 数据模型：
+
+没有标签的默认表结构：
+
+```sql
+DESC loki_demo_logs;
++--------------------+---------------------+------+------+---------+---------------+
+| Column             | Type                | Key  | Null | Default | Semantic Type |
++--------------------+---------------------+------+------+---------+---------------+
+| greptime_timestamp | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| line               | String              |      | YES  |         | FIELD         |
++--------------------+---------------------+------+------+---------+---------------+
+5 rows in set (0.00 sec)
+```
+
+- greptime_timestamp: 日志的时间戳。
+- line: 日志消息。
+
+如果您指定了外部 label，我们会将它们添加为表结构中的 tag。例如上面的 `job` 和 `from`。
+我们不能手动指定 tag，所有 lable 都被视为 tag 并且类型为字符串。请不要尝试使用 SQL 提前创建表来指定 tag 列，这会导致类型不匹配而写入失败。
+
+### 示例
+
+以下是表结构的示例：
+
+```sql
+DESC loki_demo_logs;
++--------------------+---------------------+------+------+---------+---------------+
+| Column             | Type                | Key  | Null | Default | Semantic Type |
++--------------------+---------------------+------+------+---------+---------------+
+| greptime_timestamp | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| line               | String              |      | YES  |         | FIELD         |
+| filename           | String              | PRI  | YES  |         | TAG           |
+| from               | String              | PRI  | YES  |         | TAG           |
+| job                | String              | PRI  | YES  |         | TAG           |
++--------------------+---------------------+------+------+---------+---------------+
+5 rows in set (0.00 sec)
+```
+
+```sql
+SHOW CREATE TABLE loki_demo_logs\G
+*************************** 1. row ***************************
+       Table: loki_demo_logs
+Create Table: CREATE TABLE IF NOT EXISTS `loki_demo_logs` (
+  `greptime_timestamp` TIMESTAMP(9) NOT NULL,
+  `line` STRING NULL,
+  `filename` STRING NULL,
+  `from` STRING NULL,
+  `job` STRING NULL,
+  TIME INDEX (`greptime_timestamp`),
+  PRIMARY KEY (`filename`, `from`, `job`)
+)
+
+ENGINE=mito
+WITH(
+  append_mode = 'true'
+)
+1 row in set (0.00 sec)
+```
diff --git a/...gin-content-docs/version-0.10/user-guide/ingest-data/for-observerbility/loki.md b/...gin-content-docs/version-0.10/user-guide/ingest-data/for-observerbility/loki.md
@@ -0,0 +1,122 @@
+# Loki
+
+## 使用方法
+
+### API
+
+要通过原始 HTTP API 将日志发送到 GreptimeDB，请使用以下信息：
+
+* URL: `http{s}://<host>/v1/loki/api/v1/push`
+* Headers:
+  * `X-Greptime-DB-Name`: `<dbname>`
+  * `Authorization`: `Basic` 认证，这是一个 Base64 编码的 `<username>:<password>` 字符串。更多信息，请参考 [认证](https://docs.greptime.com/user-guide/deployments/authentication/static/) 和 [HTTP API](https://docs.greptime.com/user-guide/protocols/http#authentication)。
+  * `X-Greptime-Log-Table-Name`: `<table_name>`（可选）- 存储日志的表名。如果未提供，默认表名为 `loki_logs`。
+
+请求使用二进制 protobuf 编码负载，定义的格式与 [logproto.proto](https://github.com/grafana/loki/blob/main/pkg/logproto/logproto.proto) 相同。
+
+### 示例代码
+
+[Grafana Alloy](https://grafana.com/docs/alloy/latest/) 是一个供应商中立的 OpenTelemetry (OTel) Collector 发行版。Alloy 独特地结合了社区中最好的开源可观测性信号。
+
+它提供了一个 Loki 导出器，可以用来将日志发送到 GreptimeDB。以下是一个配置示例：
+
+```hcl
+loki.source.file "greptime" {
+  targets = [
+    {__path__ = "/tmp/foo.txt"},
+  ]
+  forward_to = [loki.write.greptime_loki.receiver]
+}
+
+loki.write "greptime_loki" {
+    endpoint {
+        url = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/loki/api/v1/push"
+        headers  = {
+          "x-greptime-db-name" = "${GREPTIME_DB:=public}",
+          "x-greptime-log-table-name" = "${GREPTIME_LOG_TABLE_NAME:=loki_demo_logs}",
+        }
+    }
+    external_labels = {
+        "job" = "greptime"
+        "from" = "alloy"
+    }
+}
+```
+
+我们监听文件 `/tmp/foo.txt` 并将日志发送到 GreptimeDB。日志存储在表 `loki_demo_logs` 中，并带有 label `job` 和 `from`。
+
+更多信息，请参考 [Grafana Alloy loki.write 文档](https://grafana.com/docs/alloy/latest/reference/components/loki/loki.write/)。
+
+您可以运行以下命令来检查表中的数据：
+
+```sql
+SELECT * FROM loki_demo_logs;
++----------------------------+------------------------+--------------+-------+----------+
+| greptime_timestamp         | line                   | filename     | from  | job      |
++----------------------------+------------------------+--------------+-------+----------+
+| 2024-11-25 11:02:31.256251 | Greptime is very cool! | /tmp/foo.txt | alloy | greptime |
++----------------------------+------------------------+--------------+-------+----------+
+1 row in set (0.01 sec)
+```
+
+## 数据模型
+
+Loki 日志数据模型根据以下规则映射到 GreptimeDB 数据模型：
+
+没有标签的默认表结构：
+
+```sql
+DESC loki_demo_logs;
++--------------------+---------------------+------+------+---------+---------------+
+| Column             | Type                | Key  | Null | Default | Semantic Type |
++--------------------+---------------------+------+------+---------+---------------+
+| greptime_timestamp | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| line               | String              |      | YES  |         | FIELD         |
++--------------------+---------------------+------+------+---------+---------------+
+5 rows in set (0.00 sec)
+```
+
+- greptime_timestamp: 日志的时间戳。
+- line: 日志消息。
+
+如果您指定了外部 label，我们会将它们添加为表结构中的 tag。例如上面的 `job` 和 `from`。
+我们不能手动指定 tag，所有 lable 都被视为 tag 并且类型为字符串。请不要尝试使用 SQL 提前创建表来指定 tag 列，这会导致类型不匹配而写入失败。
+
+### 示例
+
+以下是表结构的示例：
+
+```sql
+DESC loki_demo_logs;
++--------------------+---------------------+------+------+---------+---------------+
+| Column             | Type                | Key  | Null | Default | Semantic Type |
++--------------------+---------------------+------+------+---------+---------------+
+| greptime_timestamp | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| line               | String              |      | YES  |         | FIELD         |
+| filename           | String              | PRI  | YES  |         | TAG           |
+| from               | String              | PRI  | YES  |         | TAG           |
+| job                | String              | PRI  | YES  |         | TAG           |
++--------------------+---------------------+------+------+---------+---------------+
+5 rows in set (0.00 sec)
+```
+
+```sql
+SHOW CREATE TABLE loki_demo_logs\G
+*************************** 1. row ***************************
+       Table: loki_demo_logs
+Create Table: CREATE TABLE IF NOT EXISTS `loki_demo_logs` (
+  `greptime_timestamp` TIMESTAMP(9) NOT NULL,
+  `line` STRING NULL,
+  `filename` STRING NULL,
+  `from` STRING NULL,
+  `job` STRING NULL,
+  TIME INDEX (`greptime_timestamp`),
+  PRIMARY KEY (`filename`, `from`, `job`)
+)
+
+ENGINE=mito
+WITH(
+  append_mode = 'true'
+)
+1 row in set (0.00 sec)
+```
diff --git a/versioned_docs/version-0.10/user-guide/ingest-data/for-observerbility/loki.md b/versioned_docs/version-0.10/user-guide/ingest-data/for-observerbility/loki.md
@@ -0,0 +1,123 @@
+# Loki
+
+## Usage
+
+### API
+
+To send Logs to GreptimeDB through Raw HTTP API, use the following information:
+
+* URL: `http{s}://<host>/v1/loki/api/v1/push`
+* Headers:
+  * `X-Greptime-DB-Name`: `<dbname>`
+  * `Authorization`: `Basic` authentication, which is a Base64 encoded string of `<username>:<password>`. For more information, please refer to [Authentication](https://docs.greptime.com/user-guide/deployments/authentication/static/) and [HTTP API](https://docs.greptime.com/user-guide/protocols/http#authentication).
+  * `X-Greptime-Log-Table-Name`: `<table_name>` (optional) - The table name to store the logs. If not provided, the default table name is `loki_logs`.
+
+  The request uses binary protobuf to encode the payload, The defined schema is the same as the [logproto.proto](https://github.com/grafana/loki/blob/main/pkg/logproto/logproto.proto).
+
+### Example Code
+
+[Grafana Alloy](https://grafana.com/docs/alloy/latest/) is a vendor-neutral distribution of the OpenTelemetry (OTel) Collector. Alloy uniquely combines the very best OSS observability signals in the community.
+
+It suplies a Loki exporter that can be used to send logs to GreptimeDB. Here is an example configuration:
+
+```hcl
+loki.source.file "greptime" {
+  targets = [
+    {__path__ = "/tmp/foo.txt"},
+  ]
+  forward_to = [loki.write.greptime_loki.receiver]
+}
+
+loki.write "greptime_loki" {
+    endpoint {
+        url = "${GREPTIME_SCHEME:=http}://${GREPTIME_HOST:=greptimedb}:${GREPTIME_PORT:=4000}/v1/loki/api/v1/push"
+        headers  = {
+          "x-greptime-db-name" = "${GREPTIME_DB:=public}",
+          "x-greptime-log-table-name" = "${GREPTIME_LOG_TABLE_NAME:=loki_demo_logs}",
+        }
+    }
+    external_labels = {
+        "job" = "greptime"
+        "from" = "alloy"
+    }
+}
+```
+
+We listen to the file `/tmp/foo.txt` and send the logs to GreptimeDB. The logs are stored in the table `loki_demo_logs` with the external labels `job` and `from`.
+
+For more information, please refer to the [Grafana Alloy loki.write documentation](https://grafana.com/docs/alloy/latest/reference/components/loki/loki.write/).
+
+You can run the following command to check the data in the table:
+
+```sql
+SELECT * FROM loki_demo_logs;
++----------------------------+------------------------+--------------+-------+----------+
+| greptime_timestamp         | line                   | filename     | from  | job      |
++----------------------------+------------------------+--------------+-------+----------+
+| 2024-11-25 11:02:31.256251 | Greptime is very cool! | /tmp/foo.txt | alloy | greptime |
++----------------------------+------------------------+--------------+-------+----------+
+1 row in set (0.01 sec)
+```
+
+## Data Model
+
+The Loki logs data model is mapped to the GreptimeDB data model according to the following rules:
+
+Default table schema without external labels:
+
+```sql
+DESC loki_demo_logs;
++--------------------+---------------------+------+------+---------+---------------+
+| Column             | Type                | Key  | Null | Default | Semantic Type |
++--------------------+---------------------+------+------+---------+---------------+
+| greptime_timestamp | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| line               | String              |      | YES  |         | FIELD         |
++--------------------+---------------------+------+------+---------+---------------+
+5 rows in set (0.00 sec)
+```
+
+- greptime_timestamp: The timestamp of the log.
+- line: The log message.
+
+if you specify the external labels, we will add them as tags to the table schema. like `job` and `from` in the above example.
+You can't specify tags manually, all labels are treated as tags and string type.
+Please do not attempt to pre-create the table using SQL to specify tag columns, as this will cause a type mismatch and write failure.
+
+### Example
+
+The following is an example of the table schema:
+
+```sql
+DESC loki_demo_logs;
++--------------------+---------------------+------+------+---------+---------------+
+| Column             | Type                | Key  | Null | Default | Semantic Type |
++--------------------+---------------------+------+------+---------+---------------+
+| greptime_timestamp | TimestampNanosecond | PRI  | NO   |         | TIMESTAMP     |
+| line               | String              |      | YES  |         | FIELD         |
+| filename           | String              | PRI  | YES  |         | TAG           |
+| from               | String              | PRI  | YES  |         | TAG           |
+| job                | String              | PRI  | YES  |         | TAG           |
++--------------------+---------------------+------+------+---------+---------------+
+5 rows in set (0.00 sec)
+```
+
+```sql
+SHOW CREATE TABLE loki_demo_logs\G
+*************************** 1. row ***************************
+       Table: loki_demo_logs
+Create Table: CREATE TABLE IF NOT EXISTS `loki_demo_logs` (
+  `greptime_timestamp` TIMESTAMP(9) NOT NULL,
+  `line` STRING NULL,
+  `filename` STRING NULL,
+  `from` STRING NULL,
+  `job` STRING NULL,
+  TIME INDEX (`greptime_timestamp`),
+  PRIMARY KEY (`filename`, `from`, `job`)
+)
+
+ENGINE=mito
+WITH(
+  append_mode = 'true'
+)
+1 row in set (0.00 sec)
+```