Format requirements for remote master key requests

[RotaercAH]

Description

When using external references for parameters, what are the requirements for the requested master key?
example：
SELECT pg_tde_add_key_provider_file(
'file-provider',
json_object( 'type' VALUE 'remote', 'url' VALUE 'http://localhost:8888/hello' )
);"

Suggested solution

Could you provide more specific examples in the document, or specify the requirements for the master key?

Additional context

No response

Code of Conduct

• I agree to follow this project's Code of Conduct

[dutow]

Hello!

The requirements are as for normal strings.

In the case above, the remote parameter is for the filename for the file keyring, and the URL should return a filename, where the server can store the keyring file datafile.

The URL has nothing to do with the principal key, that is still stored the same way by keyring_file in this case, in the file specified by the URL.

[RotaercAH]

Thanks for your answer.
I've come across another problem while using it.
I followed the document to install pg_tde on Percona Server for PostgreSQL
https://percona.github.io/pg_tde/main/apt.html
After executing command ALTER SYSTEM SET shared_preload_libraries = 'pg_tde';
I encountered the following error when I restarted pgsql.

root@0cca3d176e49:/# service postgresql restart

• Restarting PostgreSQL 17 database server * Error: /usr/lib/postgresql/17/bin/pg_ctl /usr/lib/postgresql/17/bin/pg_ctl start -D /var/lib/postgresql/17/main -l /var/log/postgresql/postgresql-17-main.log -s -o -c config_file="/etc/postgresql/17/main/postgresql.conf" exited with status 1:
  2024-12-04 17:49:22.834 CST [10465] FATAL: could not load library "/usr/lib/postgresql/17/lib/pg_tde.so": /usr/lib/postgresql/17/lib/pg_tde.so: undefined symbol: percona_api_version
  2024-12-04 17:49:22.834 CST [10465] LOG: database system is shut down
  pg_ctl: could not start server
  Examine the log output.
  My os is ubuntu 20.04

[ImTheKai]

Hey,

I assume you've installed upstream PostgreSQL and not our Percona Server for PostgreSQL with the needed patch, to make the full version of pg_tde work. Either you need to use upstream PG and upstream/GitHub packages or you also simply install Perconas PostgreSQL from the same experimental repo, that you've added already. Package: percona-server-17. You cannot mix upstream PostgreSQL binary, with our "downstream" pg_tde package. You need to choose.

[RotaercAH]

I've checked the software packages I installed, and it doesn't seem like there's any leftover upstream PostgreSQL. Below are the software packages I've installed.

root@0cca3d176e49:/# apt list --installed | grep postgresql

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

percona-postgresql-17-pg-tde/stable,now 1:1.0.0-1.focal amd64 [installed]
percona-postgresql-17-pgaudit/stable,now 1:17.0-1.focal amd64 [installed,automatic]
percona-postgresql-17-repack/stable,now 1:1.5.1-1.focal amd64 [installed,automatic]
percona-postgresql-17-wal2json/stable,now 1:2.6-1.focal amd64 [installed,automatic]
percona-postgresql-17/stable,now 2:17.0-1.focal amd64 [installed]
percona-postgresql-client-17/stable,now 2:17.0-1.focal amd64 [installed,automatic]
percona-postgresql-common/stable,stable,now 1:264-1.focal all [installed,automatic]
percona-postgresql-contrib/stable,now 1:264-1.focal all [installed]
percona-postgresql-server-dev-17/stable,now 2:17.0-1.focal amd64 [installed,automatic]
percona-postgresql-server-dev-all/stable,now 1:264-1.focal all [installed]
postgresql-client-common/stable,now 1:264-1.focal all [installed,automatic]
postgresql-common/stable,now 1:264-1.focal all [installed,automatic]

[ImTheKai]

Thanks for the feedback. I just rerun, and here is my list:

apt list --installed | grep postgresql

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

percona-postgresql-17-pg-tde/stable,now 1:1.0.0-20241205001625.1.noble amd64 [installed]
percona-postgresql-17/stable,now 2:17.0-1.noble amd64 [installed]
percona-postgresql-client-17/stable,now 2:17.0-1.noble amd64 [installed,automatic]
percona-postgresql-common/stable,now 1:264-1.noble all [installed,automatic]
postgresql-client-common/stable,now 1:264-1.noble all [installed,automatic]
postgresql-common/stable,now 1:264-1.noble all [installed,automatic]

Are you sure, that you didn't take the PG package from stable or another repo? I simply enabled experimental and installed server and pg_tde and both work. We're currently in the process of preparation for the beta release for tde_heap, so it might be that the documentation isn't 100% clear, and if you check Percona Server documentation, it tells you to enable stable/main/release instead of experimental. For now, the latest packages, that have everything included and are currently in testing, are the ones in the experimental repository only, as there have been changes applied to PG, and as such, you do see the API issue, with the PG build from the stable repo.

[ImTheKai]

Ah, I missed an important part from your message. You are on focal and not noble. That's the issue. We currently do not build for focal, for the latest packages, as this saves us build time. So the packages, you're getting, are missing this needed patch. Once Percona Server for PostgreSQL is released in version 17.2.1, we will also provide the focal builds.

[RotaercAH]

Based on your suggestion, I changed ubuntu from focal to noble. This device is completely new and repeated the installation steps.

After creating the container, I followed the manual and ran the following commands.

apt-get update
apt-get install -y wget gnupg2 curl lsb-release
wget https://repo.percona.com/apt/percona-release_latest.generic_all.deb
dpkg -i percona-release_latest.generic_all.deb
percona-release setup ppg-17
percona-release enable ppg-17.0 experimental
apt-get update

Here is my list:

apt list --installed | grep postgresql

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

percona-postgresql-17-pg-tde/stable,now 1:1.0.0-20241205001625.1.noble amd64 [installed]
percona-postgresql-17/stable,now 2:17.0-1.noble amd64 [installed]
percona-postgresql-client-17/stable,now 2:17.0-1.noble amd64 [installed,automatic]
percona-postgresql-common/stable,now 1:264-1.noble all [installed,automatic]
percona-postgresql-contrib/stable,now 1:264-1.noble all [installed]
percona-postgresql-server-dev-17/stable,now 2:17.0-1.noble amd64 [installed,automatic]
percona-postgresql-server-dev-all/stable,now 1:264-1.noble all [installed]
postgresql-client-common/stable,now 1:264-1.noble all [installed,automatic]
postgresql-common/stable,now 1:264-1.noble all [installed,automatic]

Then I enabled the pg_tde configuration and restarted the service, and got the same result. Have I missed any crucial steps?

postgres@1ebd51190c89:/$ psql
psql (17.0 - Percona Server for PostgreSQL 17.0.1)
Type "help" for help.

postgres=# ALTER SYSTEM SET shared_preload_libraries = 'pg_tde';

root@1ebd51190c89:/# service postgresql restart

• Restarting PostgreSQL 17 database server * Error: /usr/lib/postgresql/17/bin/pg_ctl /usr/lib/postgresql/17/bin/pg_ctl start -D /var/lib/postgresql/17/main -l /var/log/postgresql/postgresql-17-main.log -s -o -c config_file="/etc/postgresql/17/main/postgresql.conf" exited with status 1:
  2024-12-05 16:16:07.529 CST [10238] FATAL: could not load library "/usr/lib/postgresql/17/lib/pg_tde.so": /usr/lib/postgresql/17/lib/pg_tde.so: undefined symbol: percona_api_version
  2024-12-05 16:16:07.529 CST [10238] LOG: database system is shut down
  pg_ctl: could not start server
  Examine the log output.
  [fail]

[ImTheKai]

Hi,

thanks for the report. The issue is with the setup step. This will enable the "main" repository. You only need the experimental

percona-release enable ppg-17.0 experimental (That is the only command you need after the installation of percona-release)

After that install percona-server and pg_tde. That's it.

[RotaercAH]

No issues now, thanks for your reply.

[ImTheKai]

Thanks again for testing, and I will update the documentation to make things clearer.