From 893176fb5f3caeeac5beddd46f0dad1e5da85c61 Mon Sep 17 00:00:00 2001 From: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> Date: Wed, 11 Oct 2023 11:47:56 -0700 Subject: [PATCH 1/6] Updated docs for postgres and mysql components Fixes #3602 Also includes docs for https://github.com/dapr/components-contrib/pull/2975 Signed-off-by: ItalyPaleAle <43508+ItalyPaleAle@users.noreply.github.com> --- .../supported-bindings/mysql.md | 47 ++++- .../supported-bindings/postgresql.md | 79 ++++++-- .../postgresql-configuration-store.md | 189 ++++++++++++------ .../setup-postgresql.md | 91 ++++++--- 4 files changed, 279 insertions(+), 127 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md index d03dcfcab89..b7bbd9d07c5 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md @@ -55,21 +55,27 @@ Note that you can not use secret just for username/password. If you use secret, | `maxOpenConns` | N | Output | The max open connections. Integer greater than 0 | `"10"` | | `connMaxLifetime` | N | Output | The max connection lifetime. Duration string | `"12s"` | | `connMaxIdleTime` | N | Output | The max connection idel time. Duration string | `"12s"` | -| `direction` | N | Output | The direction of the binding | `"output"` | ### SSL connection If your server requires SSL your connection string must end of `&tls=custom` for example: + ```bash ":@tcp(:3306)/?allowNativePasswords=true&tls=custom" ``` - You must replace the `` with a full path to the PEM file. If you are using [MySQL on Azure](http://bit.ly/AzureMySQLSSL) see the Azure [documentation on SSL database connections](http://bit.ly/MySQLSSL), for information on how to download the required certificate. The connection to MySQL will require a minimum TLS version of 1.2. -Also note that by default [MySQL go driver](https://github.com/go-sql-driver/mysql) only supports one SQL statement per query/command. +> You must replace the `` with a full path to the PEM file. If you are using Azure Database for MySQL see the Azure [documentation on SSL database connections](https://learn.microsoft.com/azure/mysql/single-server/how-to-configure-ssl), for information on how to download the required certificate. The connection to MySQL will require a minimum TLS version of 1.2. + +### Multiple statements + +By default, the [MySQL Go driver](https://github.com/go-sql-driver/mysql) only supports one SQL statement per query/command. + To allow multiple statements in one query you need to add `multiStatements=true` to a query string, for example: + ```bash ":@tcp(:3306)/?multiStatements=true" ``` + While this allows batch queries, it also greatly increases the risk of SQL injections. Only the result of the first query is returned, all other results are silently discarded. @@ -81,17 +87,37 @@ This component supports **output binding** with the following operations: - `query` - `close` +### Parametrized queries + +This binding supports parametrized queries, which allow separating the SQL query itself from user-supplied values. The usage of parametrized queries is **strongly recommended** for security reasons, as they prevent [SQL Injection attacks](https://owasp.org/www-community/attacks/SQL_Injection). + +For example: + +```sql +-- ❌ WRONG! Includes values in the query and is vulnerable to SQL Injection attacks. +SELECT * FROM mytable WHERE user_key = 'something'; + +-- ✅ GOOD! Uses parametrized queries. +-- This will be executed with parameters ["something"] +SELECT * FROM mytable WHERE user_key = ?; +``` + +> [Relevant XKCD](https://xkcd.com/327/) + ### exec The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows). +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "exec", "metadata": { - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES (?, ?, ?)", + "params": "[1, \"demo\", \"2020-09-24T11:45:05Z07:00\"]" } } ``` @@ -106,7 +132,7 @@ The `exec` operation can be used for DDL operations (like table creation), as we "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.414519Z", "rows-affected": "1", - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES (?, ?, ?)" } } ``` @@ -115,13 +141,16 @@ The `exec` operation can be used for DDL operations (like table creation), as we The `query` operation is used for `SELECT` statements, which returns the metadata along with data in a form of an array of row values. +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "query", "metadata": { - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < $1", + "params": "[3]" } } ``` @@ -135,7 +164,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat "duration": "432µs", "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.420566Z", - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < ?" }, "data": [ {column_name: value, column_name: value, ...}, @@ -150,7 +179,7 @@ or numbers (language specific data type) ### close -Finally, the `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. +The `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. **Request** @@ -160,8 +189,6 @@ Finally, the `close` operation can be used to explicitly close the DB connection } ``` -> Note, the MySQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query. - ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md index 31c9f230cff..b4b7610ef23 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md @@ -22,10 +22,9 @@ spec: type: bindings.postgresql version: v1 metadata: - - name: url # Required - value: "" - - name: direction - value: "" + # Connection string + - name: connectionString + value: "" ``` {{% alert title="Warning" color="warning" %}} @@ -34,25 +33,48 @@ The above example uses secrets as plain strings. It is recommended to use a secr ## Spec metadata fields -| Field | Required | Binding support | Details | Example | -|--------------------|:--------:|------------|-----|---------| -| `url` | Y | Output | PostgreSQL connection string See [here](#url-format) for more details | `"user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca"` | -| `direction` | N | Output | The direction of the binding | `"output"` | +### Authenticate using a connection string + +The following metadata options are **required** to authenticate using a PostgreSQL connection string. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"` + +### Authenticate using Azure AD + +Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` | +| `connectionString` | Y | The connection string for the PostgreSQL database.
This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` | +| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` | +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` | +| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` | + +### Other metadata options + +| Field | Required | Binding support |Details | Example | +|--------------------|:--------:|-----|---|---------| +| `maxConns` | N | Output | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"` +| `connectionMaxIdleTime` | N | Output | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"` +| `queryExecMode` | N | Output | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"` ### URL format -The PostgreSQL binding uses [pgx connection pool](https://github.com/jackc/pgx) internally so the `url` parameter can be any valid connection string, either in a `DSN` or `URL` format: +The PostgreSQL binding uses [pgx connection pool](https://github.com/jackc/pgx) internally so the `connectionString` parameter can be any valid connection string, either in a `DSN` or `URL` format: **Example DSN** ```shell -user=dapr password=secret host=dapr.example.com port=5432 dbname=dapr sslmode=verify-ca +user=dapr password=secret host=dapr.example.com port=5432 dbname=my_dapr sslmode=verify-ca ``` **Example URL** ```shell -postgres://dapr:secret@dapr.example.com:5432/dapr?sslmode=verify-ca +postgres://dapr:secret@dapr.example.com:5432/my_dapr?sslmode=verify-ca ``` Both methods also support connection pool configuration variables: @@ -72,17 +94,37 @@ This component supports **output binding** with the following operations: - `query` - `close` +### Parametrized queries + +This binding supports parametrized queries, which allow separating the SQL query itself from user-supplied values. The usage of parametrized queries is **strongly recommended** for security reasons, as they prevent [SQL Injection attacks](https://owasp.org/www-community/attacks/SQL_Injection). + +For example: + +```sql +-- ❌ WRONG! Includes values in the query and is vulnerable to SQL Injection attacks. +SELECT * FROM mytable WHERE user_key = 'something'; + +-- ✅ GOOD! Uses parametrized queries. +-- This will be executed with parameters ["something"] +SELECT * FROM mytable WHERE user_key = $1; +``` + +> [Relevant XKCD](https://xkcd.com/327/) + ### exec The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows). +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "exec", "metadata": { - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES ($1, $2, $3)", + "params": "[1, \"demo\", \"2020-09-24T11:45:05Z07:00\"]" } } ``` @@ -97,7 +139,7 @@ The `exec` operation can be used for DDL operations (like table creation), as we "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.414519Z", "rows-affected": "1", - "sql": "INSERT INTO foo (id, c1, ts) VALUES (1, 'demo', '2020-09-24T11:45:05Z07:00')" + "sql": "INSERT INTO foo (id, c1, ts) VALUES ($1, $2, $3)" } } ``` @@ -106,13 +148,16 @@ The `exec` operation can be used for DDL operations (like table creation), as we The `query` operation is used for `SELECT` statements, which returns the metadata along with data in a form of an array of row values. +The `params` property is a string containing a JSON-encoded array of parameters. + **Request** ```json { "operation": "query", "metadata": { - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < $1", + "params": "[3]" } } ``` @@ -126,7 +171,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat "duration": "432µs", "start-time": "2020-09-24T11:13:46.405097Z", "end-time": "2020-09-24T11:13:46.420566Z", - "sql": "SELECT * FROM foo WHERE id < 3" + "sql": "SELECT * FROM foo WHERE id < $1" }, "data": "[ [0,\"test-0\",\"2020-09-24T04:13:46Z\"], @@ -138,7 +183,7 @@ The `query` operation is used for `SELECT` statements, which returns the metadat ### close -Finally, the `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. +The `close` operation can be used to explicitly close the DB connection and return it to the pool. This operation doesn't have any response. **Request** @@ -148,8 +193,6 @@ Finally, the `close` operation can be used to explicitly close the DB connection } ``` -> Note, the PostgreSQL binding itself doesn't prevent SQL injection, like with any database application, validate the input before executing query. - ## Related links - [Basic schema for a Dapr component]({{< ref component-schema >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md b/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md index 43b9820e081..15fa476ae00 100644 --- a/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md +++ b/daprdocs/content/en/reference/components-reference/supported-configuration-stores/postgresql-configuration-store.md @@ -21,13 +21,36 @@ spec: type: configuration.postgresql version: v1 metadata: - - name: connectionString - value: "host=localhost user=postgres password=example port=5432 connect_timeout=10 database=config" - - name: table # name of the table which holds configuration information - value: "[your_configuration_table_name]" - - name: connMaxIdleTime # max timeout for connection - value : "15s" - + # Connection string + - name: connectionString + value: "host=localhost user=postgres password=example port=5432 connect_timeout=10 database=config" + # Name of the table which holds configuration information + - name: table + value: "[your_configuration_table_name]" + # Timeout for database operations, in seconds (optional) + #- name: timeoutInSeconds + # value: 20 + # Name of the table where to store the state (optional) + #- name: tableName + # value: "state" + # Name of the table where to store metadata used by Dapr (optional) + #- name: metadataTableName + # value: "dapr_metadata" + # Cleanup interval in seconds, to remove expired rows (optional) + #- name: cleanupIntervalInSeconds + # value: 3600 + # Maximum number of connections pooled by this component (optional) + #- name: maxConns + # value: 0 + # Max idle time for connections before they're closed (optional) + #- name: connectionMaxIdleTime + # value: 0 + # Controls the default mode for executing queries. (optional) + #- name: queryExecMode + # value: "" + # Uncomment this if you wish to use PostgreSQL as a state store for actors (optional) + #- name: actorStateStore + # value: "true" ``` {{% alert title="Warning" color="warning" %}} @@ -36,69 +59,101 @@ The above example uses secrets as plain strings. It is recommended to use a secr ## Spec metadata fields -| Field | Required | Details | Example | +### Authenticate using a connection string + +The following metadata options are **required** to authenticate using a PostgreSQL connection string. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"` + +### Authenticate using Azure AD + +Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` | +| `connectionString` | Y | The connection string for the PostgreSQL database.
This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` | +| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` | +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` | +| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` | + +### Other metadata options + +| Field | Required | Details | Example | |--------------------|:--------:|---------|---------| -| connectionString | Y | The connection string for PostgreSQL. Default pool_max_conns = 5 | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test pool_max_conns=10"` -| table | Y | Table name for configuration information, must be lowercased. | `configtable` +| `table` | Y | Table name for configuration information, must be lowercased. | `configtable` +| `maxConns` | N | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"` +| `connectionMaxIdleTime` | N | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"` +| `queryExecMode` | N | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"` ## Set up PostgreSQL as Configuration Store -1. Start PostgreSQL Database -1. Connect to the PostgreSQL database and setup a configuration table with following schema - +1. Start the PostgreSQL Database +1. Connect to the PostgreSQL database and setup a configuration table with following schema: + + | Field | Datatype | Nullable |Details | + |--------------------|:--------:|---------|---------| + | KEY | VARCHAR | N |Holds `"Key"` of the configuration attribute | + | VALUE | VARCHAR | N |Holds Value of the configuration attribute | + | VERSION | VARCHAR | N | Holds version of the configuration attribute | + | METADATA | JSON | Y | Holds Metadata as JSON | + + ```sql + CREATE TABLE IF NOT EXISTS table_name ( + KEY VARCHAR NOT NULL, + VALUE VARCHAR NOT NULL, + VERSION VARCHAR NOT NULL, + METADATA JSON + ); + ``` + +3. Create a TRIGGER on configuration table. An example function to create a TRIGGER is as follows: + + ```sh + CREATE OR REPLACE FUNCTION configuration_event() RETURNS TRIGGER AS $$ + DECLARE + data json; + notification json; + + BEGIN + + IF (TG_OP = 'DELETE') THEN + data = row_to_json(OLD); + ELSE + data = row_to_json(NEW); + END IF; + + notification = json_build_object( + 'table',TG_TABLE_NAME, + 'action', TG_OP, + 'data', data); + PERFORM pg_notify('config',notification::text); + RETURN NULL; + END; + $$ LANGUAGE plpgsql; + ``` + +4. Create the trigger with data encapsulated in the field labeled as `data`: + + ```sql + notification = json_build_object( + 'table',TG_TABLE_NAME, + 'action', TG_OP, + 'data', data + ); + ``` -| Field | Datatype | Nullable |Details | -|--------------------|:--------:|---------|---------| -| KEY | VARCHAR | N |Holds `"Key"` of the configuration attribute | -| VALUE | VARCHAR | N |Holds Value of the configuration attribute | -| VERSION | VARCHAR | N | Holds version of the configuration attribute -| METADATA | JSON | Y | Holds Metadata as JSON - -```console -CREATE TABLE IF NOT EXISTS table_name ( - KEY VARCHAR NOT NULL, - VALUE VARCHAR NOT NULL, - VERSION VARCHAR NOT NULL, - METADATA JSON ); -``` -3. Create a TRIGGER on configuration table. An example function to create a TRIGGER is as follows - -```console -CREATE OR REPLACE FUNCTION configuration_event() RETURNS TRIGGER AS $$ - DECLARE - data json; - notification json; - - BEGIN - - IF (TG_OP = 'DELETE') THEN - data = row_to_json(OLD); - ELSE - data = row_to_json(NEW); - END IF; - - notification = json_build_object( - 'table',TG_TABLE_NAME, - 'action', TG_OP, - 'data', data); - - PERFORM pg_notify('config',notification::text); - RETURN NULL; - END; -$$ LANGUAGE plpgsql; -``` -4. Create the trigger with data encapsulated in the field labelled as `data` -```ps -notification = json_build_object( - 'table',TG_TABLE_NAME, - 'action', TG_OP, - 'data', data); -``` 5. The channel mentioned as attribute to `pg_notify` should be used when subscribing for configuration notifications 6. Since this is a generic created trigger, map this trigger to `configuration table` -```console -CREATE TRIGGER config -AFTER INSERT OR UPDATE OR DELETE ON configtable - FOR EACH ROW EXECUTE PROCEDURE notify_event(); -``` + + ```sql + CREATE TRIGGER config + AFTER INSERT OR UPDATE OR DELETE ON configtable + FOR EACH ROW EXECUTE PROCEDURE notify_event(); + ``` + 7. In the subscribe request add an additional metadata field with key as `pgNotifyChannel` and value should be set to same `channel name` mentioned in `pg_notify`. From the above example, it should be set to `config` {{% alert title="Note" color="primary" %}} @@ -106,12 +161,14 @@ When calling `subscribe` API, `metadata.pgNotifyChannel` should be used to speci Any number of keys can be added to a subscription request. Each subscription uses an exclusive database connection. It is strongly recommended to subscribe to multiple keys within a single subscription. This helps optimize the number of connections to the database. -Example of subscribe HTTP API - -```ps -curl --location --request GET 'http://:/configuration/mypostgresql/subscribe?key=&key=&metadata.pgNotifyChannel=' +Example of subscribe HTTP API: + +```sh +curl -l 'http://:/configuration/mypostgresql/subscribe?key=&key=&metadata.pgNotifyChannel=' ``` {{% /alert %}} ## Related links + - [Basic schema for a Dapr component]({{< ref component-schema >}}) - [Configuration building block]({{< ref configuration-api-overview >}}) diff --git a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md index 6e1bfad9216..0d5c682422e 100644 --- a/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-state-stores/setup-postgresql.md @@ -7,13 +7,7 @@ aliases: - "/operations/components/setup-state-store/supported-state-stores/setup-postgresql/" --- -This component allows using PostgreSQL (Postgres) as state store for Dapr. - -## Create a Dapr component - -Create a file called `postgresql.yaml`, paste the following and replace the `` value with your connection string. The connection string is a standard PostgreSQL connection string. For example, `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test"`. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. - -If you want to also configure PostgreSQL to store actors, add the `actorStateStore` option as in the example below. +This component allows using PostgreSQL (Postgres) as state store for Dapr. See [this guide]({{< ref "howto-get-save-state.md#step-1-setup-a-state-store" >}}) on how to create and apply a state store configuration. ```yaml apiVersion: dapr.io/v1alpha1 @@ -24,42 +18,72 @@ spec: type: state.postgresql version: v1 metadata: - # Connection string - - name: connectionString - value: "" - # Timeout for database operations, in seconds (optional) - #- name: timeoutInSeconds - # value: 20 - # Name of the table where to store the state (optional) - #- name: tableName - # value: "state" - # Name of the table where to store metadata used by Dapr (optional) - #- name: metadataTableName - # value: "dapr_metadata" - # Cleanup interval in seconds, to remove expired rows (optional) - #- name: cleanupIntervalInSeconds - # value: 3600 - # Max idle time for connections before they're closed (optional) - #- name: connectionMaxIdleTime - # value: 0 - # Uncomment this if you wish to use PostgreSQL as a state store for actors (optional) - #- name: actorStateStore - # value: "true" + # Connection string + - name: connectionString + value: "" + # Timeout for database operations, in seconds (optional) + #- name: timeoutInSeconds + # value: 20 + # Name of the table where to store the state (optional) + #- name: tableName + # value: "state" + # Name of the table where to store metadata used by Dapr (optional) + #- name: metadataTableName + # value: "dapr_metadata" + # Cleanup interval in seconds, to remove expired rows (optional) + #- name: cleanupIntervalInSeconds + # value: 3600 + # Maximum number of connections pooled by this component (optional) + #- name: maxConns + # value: 0 + # Max idle time for connections before they're closed (optional) + #- name: connectionMaxIdleTime + # value: 0 + # Controls the default mode for executing queries. (optional) + #- name: queryExecMode + # value: "" + # Uncomment this if you wish to use PostgreSQL as a state store for actors (optional) + #- name: actorStateStore + # value: "true" ``` + {{% alert title="Warning" color="warning" %}} The above example uses secrets as plain strings. It is recommended to use a secret store for the secrets as described [here]({{< ref component-secrets.md >}}). {{% /alert %}} ## Spec metadata fields +### Authenticate using a connection string + +The following metadata options are **required** to authenticate using a PostgreSQL connection string. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `connectionString` | Y | The connection string for the PostgreSQL database. See the PostgreSQL [documentation on database connections](https://www.postgresql.org/docs/current/libpq-connect.html) for information on how to define a connection string. | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=my_db"` + +### Authenticate using Azure AD + +Authenticating with Azure AD is supported with Azure Database for PostgreSQL. All authentication methods supported by Dapr can be used, including client credentials ("service principal") and Managed Identity. + +| Field | Required | Details | Example | +|--------|:--------:|---------|---------| +| `useAzureAD` | Y | Must be set to `true` to enable the component to retrieve access tokens from Azure AD. | `"true"` | +| `connectionString` | Y | The connection string for the PostgreSQL database.
This must contain the user, which corresponds to the name of the user created inside PostgreSQL that maps to the Azure AD identity; this is often the name of the corresponding principal (e.g. the name of the Azure AD application). This connection string should not contain any password. | `"host=mydb.postgres.database.azure.com user=myapplication port=5432 database=my_db sslmode=require"` | +| `azureTenantId` | N | ID of the Azure AD tenant | `"cd4b2887-304c-…"` | +| `azureClientId` | N | Client ID (application ID) | `"c7dd251f-811f-…"` | +| `azureClientSecret` | N | Client secret (application password) | `"Ecy3X…"` | + +### Other metadata options + | Field | Required | Details | Example | |--------------------|:--------:|---------|---------| -| `connectionString` | Y | The connection string for the PostgreSQL database | `"host=localhost user=postgres password=example port=5432 connect_timeout=10 database=dapr_test"` | `timeoutInSeconds` | N | Timeout, in seconds, for all database operations. Defaults to `20` | `30` | `tableName` | N | Name of the table where the data is stored. Defaults to `state`. Can optionally have the schema name as prefix, such as `public.state` | `"state"`, `"public.state"` | `metadataTableName` | N | Name of the table Dapr uses to store a few metadata properties. Defaults to `dapr_metadata`. Can optionally have the schema name as prefix, such as `public.dapr_metadata` | `"dapr_metadata"`, `"public.dapr_metadata"` | `cleanupIntervalInSeconds` | N | Interval, in seconds, to clean up rows with an expired TTL. Default: `3600` (i.e. 1 hour). Setting this to values <=0 disables the periodic cleanup. | `1800`, `-1` +| `maxConns` | N | Maximum number of connections pooled by this component. Set to 0 or lower to use the default value, which is the greater of 4 or the number of CPUs. | `"4"` | `connectionMaxIdleTime` | N | Max idle time before unused connections are automatically closed in the connection pool. By default, there's no value and this is left to the database driver to choose. | `"5m"` +| `queryExecMode` | N | Controls the default mode for executing queries. By default Dapr uses the extended protocol and automatically prepares and caches prepared statements. However, this may be incompatible with proxies such as PGBouncer. In this case it may be preferrable to use `exec` or `simple_protocol`. | `"simple_protocol"` | `actorStateStore` | N | Consider this state store for actors. Defaults to `"false"` | `"true"`, `"false"` ## Setup PostgreSQL @@ -70,20 +94,21 @@ The above example uses secrets as plain strings. It is recommended to use a secr 1. Run an instance of PostgreSQL. You can run a local instance of PostgreSQL in Docker CE with the following command: - This example does not describe a production configuration because it sets the password in plain text and the user name is left as the PostgreSQL default of "postgres". - ```bash docker run -p 5432:5432 -e POSTGRES_PASSWORD=example postgres ``` + > This example does not describe a production configuration because it sets the password in plain text and the user name is left as the PostgreSQL default of "postgres". + 2. Create a database for state data. Either the default "postgres" database can be used, or create a new database for storing state data. To create a new database in PostgreSQL, run the following SQL command: - ```SQL - CREATE DATABASE dapr_test; + ```sql + CREATE DATABASE my_dapr; ``` + {{% /codetab %}} {{% /tabs %}} From b1b3b6d8ac175156af4a107f155efcdb5e65bf5d Mon Sep 17 00:00:00 2001 From: Artur Souza Date: Wed, 11 Oct 2023 14:49:44 -0700 Subject: [PATCH 2/6] Update mysql.md --- .../reference/components-reference/supported-bindings/mysql.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md index b7bbd9d07c5..0588fe75b60 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md @@ -102,8 +102,6 @@ SELECT * FROM mytable WHERE user_key = 'something'; SELECT * FROM mytable WHERE user_key = ?; ``` -> [Relevant XKCD](https://xkcd.com/327/) - ### exec The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows). From e9f22d4e5ee6921f6e0103f881deca8e23c46e25 Mon Sep 17 00:00:00 2001 From: Artur Souza Date: Wed, 11 Oct 2023 14:50:21 -0700 Subject: [PATCH 3/6] Update postgresql.md --- .../components-reference/supported-bindings/postgresql.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md index b4b7610ef23..0a21d93b663 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md @@ -109,8 +109,6 @@ SELECT * FROM mytable WHERE user_key = 'something'; SELECT * FROM mytable WHERE user_key = $1; ``` -> [Relevant XKCD](https://xkcd.com/327/) - ### exec The `exec` operation can be used for DDL operations (like table creation), as well as `INSERT`, `UPDATE`, `DELETE` operations which return only metadata (e.g. number of affected rows). From e4ffd564f1e3fdb1e60a30f16eaf32e25b0da16a Mon Sep 17 00:00:00 2001 From: Mark Fussell Date: Wed, 11 Oct 2023 14:59:36 -0700 Subject: [PATCH 4/6] Update daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md Signed-off-by: Mark Fussell --- .../reference/components-reference/supported-bindings/mysql.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md index 0588fe75b60..00b92d8acf9 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md @@ -64,7 +64,7 @@ If your server requires SSL your connection string must end of `&tls=custom` for ":@tcp(:3306)/?allowNativePasswords=true&tls=custom" ``` -> You must replace the `` with a full path to the PEM file. If you are using Azure Database for MySQL see the Azure [documentation on SSL database connections](https://learn.microsoft.com/azure/mysql/single-server/how-to-configure-ssl), for information on how to download the required certificate. The connection to MySQL will require a minimum TLS version of 1.2. +> You must replace the `` with a full path to the PEM file. If you are using Azure Database for MySQL see the Azure [documentation on SSL database connections](https://learn.microsoft.com/azure/mysql/single-server/how-to-configure-ssl), for information on how to download the required certificate. The connection to MySQL requires a minimum TLS version of 1.2. ### Multiple statements From 67416b3fbe52893b4c2af9d1c5d38c7130ad6330 Mon Sep 17 00:00:00 2001 From: Mark Fussell Date: Wed, 11 Oct 2023 15:00:25 -0700 Subject: [PATCH 5/6] Update postgresql.md --- .../components-reference/supported-bindings/postgresql.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md index 0a21d93b663..cfaf92ad37c 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/postgresql.md @@ -25,6 +25,8 @@ spec: # Connection string - name: connectionString value: "" + - name: direction + value: "" ``` {{% alert title="Warning" color="warning" %}} From 379a0ce00ca248c760c448c28ced6cafaa96a055 Mon Sep 17 00:00:00 2001 From: Mark Fussell Date: Wed, 11 Oct 2023 15:01:00 -0700 Subject: [PATCH 6/6] Update mysql.md --- .../reference/components-reference/supported-bindings/mysql.md | 1 + 1 file changed, 1 insertion(+) diff --git a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md index 00b92d8acf9..3c44b53a84c 100644 --- a/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md +++ b/daprdocs/content/en/reference/components-reference/supported-bindings/mysql.md @@ -55,6 +55,7 @@ Note that you can not use secret just for username/password. If you use secret, | `maxOpenConns` | N | Output | The max open connections. Integer greater than 0 | `"10"` | | `connMaxLifetime` | N | Output | The max connection lifetime. Duration string | `"12s"` | | `connMaxIdleTime` | N | Output | The max connection idel time. Duration string | `"12s"` | +| `direction` | N | Output | The direction of the binding | `"output"` | ### SSL connection