Postgres upgrade to v14

[bartversluijs]

Hi,

When updating the container from v5.1.0 to v5.1.1 (which upgrades to v14), the automatic upgrader won't work because it can't remove the /var/lib/postgres/data/13 folder. It'll crash, boot up again and works then. However, the /var/lib/postgres/data/13 still exists.

I'm trying to debug it and probably have a solution, however this means it'll run the delete_old_cluster.sh as root. Probably not ideal, but worth a try.

I'll create a PR for convenience, but if this isn't the best solution, please let me know!

The log:

root@openbalena-db-0:/# /balena-entrypoint.sh postgres
=== Upgrading data from v13 to v14
=== Creating postgres directory /usr/lib/postgresql/upgrade
=== Installing tools for Postgres v13
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package postgresql-client-13.
(Reading database ... 12109 files and directories currently installed.)
Preparing to unpack .../postgresql-client-13_13.7-1.pgdg110+1_amd64.deb ...
Unpacking postgresql-client-13 (13.7-1.pgdg110+1) ...
Selecting previously unselected package postgresql-13.
Preparing to unpack .../postgresql-13_13.7-1.pgdg110+1_amd64.deb ...
Unpacking postgresql-13 (13.7-1.pgdg110+1) ...
Setting up postgresql-client-13 (13.7-1.pgdg110+1) ...
Setting up postgresql-13 (13.7-1.pgdg110+1) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78.)
debconf: falling back to frontend: Readline
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Processing triggers for postgresql-common (241.pgdg110+1) ...
debconf: unable to initialize frontend: Dialog
debconf: (No usable dialog-like program is installed, so the dialog based frontend cannot be used. at /usr/share/perl5/Debconf/FrontEnd/Dialog.pm line 78.)
debconf: falling back to frontend: Readline
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
=== Initializing new data directory /var/lib/postgresql/data/14
=== Creating postgres directory /var/lib/postgresql/data/14
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory /var/lib/postgresql/data/14 ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    pg_ctl -D /var/lib/postgresql/data/14 -l logfile start

=== Running pg_upgrade
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for system-defined composite types in user tables  ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for user-defined encoding conversions              ok
Checking for user-defined postfix operators                 ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok
Checking for new cluster tablespace directories             ok

*Clusters are compatible*
Performing Consistency Checks
-----------------------------
Checking cluster versions                                   ok
Checking database user is the install user                  ok
Checking database connection settings                       ok
Checking for prepared transactions                          ok
Checking for system-defined composite types in user tables  ok
Checking for reg* data types in user tables                 ok
Checking for contrib/isn with bigint-passing mismatch       ok
Checking for user-defined encoding conversions              ok
Checking for user-defined postfix operators                 ok
Creating dump of global objects                             ok
Creating dump of database schemas
                                                            ok
Checking for presence of required libraries                 ok
Checking database user is the install user                  ok
Checking for prepared transactions                          ok
Checking for new cluster tablespace directories             ok

If pg_upgrade fails after this point, you must re-initdb the
new cluster before continuing.

Performing Upgrade
------------------
Analyzing all rows in the new cluster                       ok
Freezing all rows in the new cluster                        ok
Deleting files from new pg_xact                             ok
Copying old pg_xact to new server                           ok
Setting oldest XID for new cluster                          ok
Setting next transaction ID and epoch for new cluster       ok
Deleting files from new pg_multixact/offsets                ok
Copying old pg_multixact/offsets to new server              ok
Deleting files from new pg_multixact/members                ok
Copying old pg_multixact/members to new server              ok
Setting next multixact ID and offset for new cluster        ok
Resetting WAL archives                                      ok
Setting frozenxid and minmxid counters in new cluster       ok
Restoring global objects in the new cluster                 ok
Restoring database schemas in the new cluster
                                                            ok
Copying user relation files
                                                            ok
Setting next OID for new cluster                            ok
Sync data directory to disk                                 ok
Creating script to delete old cluster                       ok
Checking for extension updates                              notice

Your installation contains extensions that should be updated
with the ALTER EXTENSION command.  The file
    update_extensions.sql
when executed by psql by the database superuser will update
these extensions.


Upgrade Complete
----------------
Optimizer statistics are not transferred by pg_upgrade.
Once you start the new server, consider running:
    /usr/lib/postgresql/14/bin/vacuumdb -U docker --all --analyze-in-stages

Running this script will delete the old cluster's data files:
    ./delete_old_cluster.sh
=== Restoring configuration files
=== Data upgraded successfully
=== Performing post-upgrade steps
=== Starting temp server
waiting for server to start....2022-06-20 18:36:18.236 UTC [1666] LOG:  starting PostgreSQL 14.3 (Debian 14.3-1.pgdg110+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 10.2.1-6) 10.2.1 20210110, 64-bit
2022-06-20 18:36:18.239 UTC [1666] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
2022-06-20 18:36:18.251 UTC [1667] LOG:  database system was shut down at 2022-06-20 18:36:18 UTC
2022-06-20 18:36:18.270 UTC [1666] LOG:  database system is ready to accept connections
 done
server started
=== Updating extensions
You are now connected to database "resin" as user "docker".
ALTER EXTENSION
=== Deleting old cluster
rm: cannot remove '/var/lib/postgresql/data/13': Permission denied

[bartversluijs]

PR #36 fixes this, but needs review.

[bartversluijs]

Tested the latest version (v5.1.2) and the upgrade works as expected. Closing this issue.