Skip to content

Commit

Permalink
docs: update prometheus remote write document (#834)
Browse files Browse the repository at this point in the history 
Signed-off-by: Ruihang Xia <[email protected]>
Co-authored-by: Yiran <[email protected]>
  • Loading branch information
@waynexia @nicecui
waynexia and nicecui authored Apr 15, 2024
1 parent 530c943 commit 032f5eb
Show file tree
Hide file tree
Showing 4 changed files with 151 additions and 115 deletions.
74 changes: 38 additions & 36 deletions docs/v0.7/en/user-guide/operations/configuration.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -182,35 +182,37 @@ enable = true

[prom_store]
enable = true
with_metric_engine = true
```

The following table describes the options in detail:

| Option | Key | Type | Description |
| ---------- | ------------ | ------- | ------------------------------------------------------------------------------- |
| http | | | HTTP server options |
| | addr | String | Server address, "127.0.0.1:4000" by default |
| | timeout | String | HTTP request timeout, "30s" by default |
| | body_limit | String | HTTP max body size, "64MB" by default |
| grpc | | | gRPC server options |
| | addr | String | Server address, "127.0.0.1:4001" by default |
| | runtime_size | Integer | The number of server worker threads, 8 by default |
| mysql | | | MySQL server options |
| | enable | Boolean | Whether to enable MySQL protocol, true by default |
| | add | String | Server address, "127.0.0.1:4002" by default |
| | runtime_size | Integer | The number of server worker threads, 2 by default |
| influxdb | | | InfluxDB Protocol options |
| | enable | Boolean | Whether to enable InfluxDB protocol in HTTP API, true by default |
| opentsdb | | | OpenTSDB Protocol options |
| | enable | Boolean | Whether to enable OpenTSDB protocol, true by default |
| | addr | String | OpenTSDB telnet API server address, "127.0.0.1:4242" by default |
| | runtime_size | Integer | The number of server worker threads, 2 by default |
| prom_store | | | Prometheus remote storage options |
| | enable | Boolean | Whether to enable Prometheus remote write and read in HTTP API, true by default |
| postgres | | | PostgresSQL server options |
| | enable | Boolean | Whether to enable PostgresSQL protocol, true by default |
| | addr | String | Server address, "127.0.0.1:4003" by default |
| | runtime_size | Integer | The number of server worker threads, 2 by default |
| Option | Key | Type | Description |
| ---------- | ------------------ | ------- | ------------------------------------------------------------------------------- |
| http | | | HTTP server options |
| | addr | String | Server address, "127.0.0.1:4000" by default |
| | timeout | String | HTTP request timeout, "30s" by default |
| | body_limit | String | HTTP max body size, "64MB" by default |
| grpc | | | gRPC server options |
| | addr | String | Server address, "127.0.0.1:4001" by default |
| | runtime_size | Integer | The number of server worker threads, 8 by default |
| mysql | | | MySQL server options |
| | enable | Boolean | Whether to enable MySQL protocol, true by default |
| | add | String | Server address, "127.0.0.1:4002" by default |
| | runtime_size | Integer | The number of server worker threads, 2 by default |
| influxdb | | | InfluxDB Protocol options |
| | enable | Boolean | Whether to enable InfluxDB protocol in HTTP API, true by default |
| opentsdb | | | OpenTSDB Protocol options |
| | enable | Boolean | Whether to enable OpenTSDB protocol, true by default |
| | addr | String | OpenTSDB telnet API server address, "127.0.0.1:4242" by default |
| | runtime_size | Integer | The number of server worker threads, 2 by default |
| prom_store | | | Prometheus remote storage options |
| | enable | Boolean | Whether to enable Prometheus Remote Write and read in HTTP API, true by default |
| | with_metric_engine | Boolean | Whether to use the metric engine on Prometheus Remote Write, true by default |
| postgres | | | PostgresSQL server options |
| | enable | Boolean | Whether to enable PostgresSQL protocol, true by default |
| | addr | String | Server address, "127.0.0.1:4003" by default |
| | runtime_size | Integer | The number of server worker threads, 2 by default |

### Storage options

Expand All @@ -221,7 +223,7 @@ GreptimeDB supports storing data in local file system, AWS S3 and compatible ser
| Option | Key | Type | Description |
| ------- | ----------------- | ------ | ------------------------------------------------------------- |
| storage | | | Storage options |
| | type | String | Storage type, supports "File", "S3" and "Oss" etc. |
| | type | String | Storage type, supports "File", "S3" and "Oss" etc. |
| File | | | Local file storage options, valid when type="File" |
| | data_home | String | Database storage root directory, "/tmp/greptimedb" by default |
| S3 | | | AWS S3 storage options, valid when type="S3" |
Expand Down Expand Up @@ -484,11 +486,11 @@ use_memory_store = false

| Key | Type | Description |
| ---------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| data_home | String | The working home of Metasrv, `"/tmp/metasrv/"` by default |
| bind_addr | String | The bind address of Metasrv, `"127.0.0.1:3002"` by default. |
| server_addr | String | The communication server address for frontend and datanode to connect to Metasrv, `"127.0.0.1:3002"` by default for localhost |
| data_home | String | The working home of Metasrv, `"/tmp/metasrv/"` by default |
| bind_addr | String | The bind address of Metasrv, `"127.0.0.1:3002"` by default. |
| server_addr | String | The communication server address for frontend and datanode to connect to Metasrv, `"127.0.0.1:3002"` by default for localhost |
| store_addr | String | etcd server addresses, `"127.0.0.1:2379"` by default, server address separated by commas, in the format of `"ip1:port1,ip2:port2,..."`. |
| selector | String | Load balance strategy to choose datanode when creating new tables, see [Selector](/contributor-guide/metasrv/selector.md) |
| selector | String | Load balance strategy to choose datanode when creating new tables, see [Selector](/contributor-guide/metasrv/selector.md) |
| use_memory_store | Boolean | Only used for testing when you don't have an etcd cluster, store data in memory, `false` by default. |

### Datanode-only configuration
Expand All @@ -500,12 +502,12 @@ rpc_addr = "127.0.0.1:3001"
rpc_runtime_size = 8
```

| Key | Type | Description |
| ---------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| node_id | Integer | The datanode identifier, should be unique. |
| rpc_hostname | String | Hostname of this node. |
| rpc_addr | String | gRPC server address, `"127.0.0.1:3001"` by default. |
| rpc_runtime_size | Integer | The number of gRPC server worker threads, 8 by default. |
| Key | Type | Description |
| ---------------- | ------- | ------------------------------------------------------- |
| node_id | Integer | The datanode identifier, should be unique. |
| rpc_hostname | String | Hostname of this node. |
| rpc_addr | String | gRPC server address, `"127.0.0.1:3001"` by default. |
| rpc_runtime_size | Integer | The number of gRPC server worker threads, 8 by default. |

## Environment variable

Expand Down
31 changes: 24 additions & 7 deletions docs/v0.7/en/user-guide/write-data/prometheus.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ To configure Prometheus, use the following settings in the [Prometheus configura

```yaml
remote_write:
- url: http://localhost:4000/v1/prometheus/write?db=public
- url: http://localhost:4000/v1/prometheus/write?db=public&physical_table=greptime_physical_table
# basic_auth:
# username: greptime_user
# password: greptime_pwd
Expand All @@ -22,11 +22,28 @@ remote_read:
Be sure to uncomment `basic_auth` section and replace `greptime_user(username)`, `greptime_pwd(password)` with your own username and password when you enable database authentication. Please refer to client [authentication](../clients/authentication.md).
:::

The `db` parameter in the url represents the database that we want to write data.
The `db` parameter in the URL represents the database that we want to write data. It's optional.
By default, the database is `public`.
If you want to write to another database, you can [create a new database](../table-management.md#create-database)
and replace `public` with the new database name.

GreptimeDB automatically groups multiple Prometheus metrics (../clients/prometheus#data-model) into the corresponding logical tables, so you do not need to specify the logical table in the URL of `remote_write`.

The optional `physical_table` parameter in the URL represents the [physical table](/contributor-guide/datanode/metric-engine#physical-table),
which reduces storage overhead for small tables and improves columnar compression efficiency.
If unspecified, the `greptime_physical_table` is used by default;
if the specified physical table does not exist, it will be automatically created.

The `physical_table` parameter only works when the [`with_metric_engine`](/user-guide/operations/configuration.md#protocol-options) is enabled in the configuration file.
It is enabled by default.

Here is a table of URL parameters:

| Param | Required | Default Value |
| -------------- | -------- | ----------------------- |
| db | optional | public |
| physical_table | optional | greptime_physical_table |

Show tables when writing successfully:

```sql
Expand Down Expand Up @@ -54,12 +71,12 @@ show tables;
When the metrics are written into GreptimeDB by remote write endpoint, they will be transformed as
follows:

| Sample Metrics | In GreptimeDB | GreptimeDB Data Types |
|:---------------|:-----------------------------|:----------------------|
| Name | Table (Auto-created) Name | String |
| Sample Metrics | In GreptimeDB | GreptimeDB Data Types |
| :------------- | :-------------------------- | :-------------------- |
| Name | Table (Auto-created) Name | String |
| Value | Column (greptime_value) | Double |
| Timestamp | Column (greptime_timestamp) | Timestamp |
| Label | Column | String |
| Label | Column | String |

A primary key with all label columns will be created automatically. When a new label is added, it
will be added into primary key automatically too.
Expand All @@ -74,7 +91,7 @@ remote_name="648f0c", url="http://localhost:4000/v1/prometheus/write"} 500
This example will be transformed as a row in the table `prometheus_remote_storage_samples_total`：

| Column | Value | Column Data Type |
|:-------------------|:--------------------------------------------|:-------------------|
| :----------------- | :------------------------------------------ | :----------------- |
| instance | localhost:9090 | String |
| job | prometheus | String |
| remote_name | 648f0c | String |
Expand Down
Loading

0 comments on commit 032f5eb

Please sign in to comment.