How to solve Error establishing a database connection

[rishabhsdev]

I was getting Error establishing a database connection on my wordpress website built with easyengine.

I ran this comand: df -h and found that my storage was full. So, I cleared some old log files.

But still I was getting Error establishing a database connection error

So I tried running docker ps command and this was the result:

CONTAINER ID   IMAGE                           COMMAND                  CREATED       STATUS                          PORTS                                                                      NAMES
*********   easyengine/cron:v4.0.0          "/usr/bin/ofelia dae…"   2 weeks ago   Up 10 minutes                                                                                              ee-cron-scheduler
*********   easyengine/nginx:v4.1.4         "/usr/bin/openresty …"   2 weeks ago   Up 10 minutes                   80/tcp                                                                     9to6livein_nginx_1
*********   easyengine/postfix:v4.1.5       "postfix start-fg"       2 weeks ago   Up 10 minutes                   25/tcp                                                                     9to6livein_postfix_1
*********   easyengine/php:v4.1.6           "docker-entrypoint.s…"   2 weeks ago   Up 10 minutes                   9000/tcp                                                                   9to6livein_php_1
*********   easyengine/mariadb:v4.1.3       "docker-entrypoint.s…"   2 weeks ago   Restarting (1) 37 seconds ago                                                                              services_global-db_1
*********   easyengine/nginx-proxy:v4.1.4   "/app/docker-entrypo…"   2 weeks ago   Up 6 minutes                    0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   services_global-nginx-proxy_1

As you can see the mariadb container is continuously restarting. How do I fix this?

Here's the log file of docker logs services_global-db_1:

2021-04-16 11:09:28+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.4+maria~focal started.
2021-04-16 11:09:29+00:00 [Note] [Entrypoint]: Switching to dedicated user 'mysql'
2021-04-16 11:09:29+00:00 [Note] [Entrypoint]: Entrypoint script for MySQL Server 1:10.5.4+maria~focal started.
2021-04-16 11:09:29 0 [Note] mysqld (mysqld 10.5.4-MariaDB-1:10.5.4+maria~focal) starting as process 1 ...
2021-04-16 11:09:29 0 [Warning] You need to use --log-bin to make --binlog-format work.
2021-04-16 11:09:29 0 [Note] InnoDB: Using Linux native AIO
2021-04-16 11:09:29 0 [Note] InnoDB: Uses event mutexes
2021-04-16 11:09:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-04-16 11:09:29 0 [Note] InnoDB: Number of pools: 1
2021-04-16 11:09:29 0 [Note] InnoDB: Using SSE4.2 crc32 instructions
2021-04-16 11:09:29 0 [Note] mysqld: O_TMPFILE is not supported on /tmp (disabling future attempts)
2021-04-16 11:09:30 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728
2021-04-16 11:09:30 0 [Note] InnoDB: Completed initialization of buffer pool
2021-04-16 11:09:30 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority().
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955232 was in the XA prepared state.
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955166 was in the XA prepared state.
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955167 was in the XA prepared state.
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955173 was in the XA prepared state.
2021-04-16 11:09:30 0 [Note] InnoDB: 4 transaction(s) which must be rolled back or cleaned up in total 0 row operations to undo
2021-04-16 11:09:30 0 [Note] InnoDB: Trx id counter is 955233
2021-04-16 11:09:30 0 [Note] InnoDB: 128 rollback segments are active.
2021-04-16 11:09:30 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-04-16 11:09:30 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-04-16 11:09:30 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-04-16 11:09:30 0 [Note] InnoDB: 10.5.4 started; log sequence number 3046186820; transaction id 955235
2021-04-16 11:09:30 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-04-16 11:09:30 0 [Note] InnoDB: Loading buffer pool(s) from /var/lib/mysql/ib_buffer_pool
2021-04-16 11:09:30 0 [Note] InnoDB: Buffer pool(s) load completed at 210416 11:09:30
2021-04-16 11:09:30 0 [Note] InnoDB: Starting in background the rollback of recovered transactions
2021-04-16 11:09:30 0 [Note] InnoDB: Starting recovery for XA transactions...
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955167 in prepared state after recovery
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction contains changes to 1 rows
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955232 in prepared state after recovery
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction contains changes to 1 rows
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955173 in prepared state after recovery
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction contains changes to 1 rows
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction 955166 in prepared state after recovery
2021-04-16 11:09:30 0 [Note] InnoDB: Transaction contains changes to 1 rows
2021-04-16 11:09:30 0 [Note] InnoDB: 4 transactions in prepared state after recovery
2021-04-16 11:09:30 0 [Note] Found 4 prepared transaction(s) in InnoDB
2021-04-16 11:09:30 0 [ERROR] Found 4 prepared transactions! It means that mysqld was not shut down properly last time and critical recovery information (last binlog or tc.log file) was manually deleted after a crash. You have to start mysqld with --tc-heuristic-recover switch to commit or rollback pending transactions.
2021-04-16 11:09:30 0 [ERROR] Aborting

How to solve this?

[dhsathiya]

Did you delete anything from /opt/easyengine/services/mariadb/logs ?

[rishabhsdev]

No, I didn't delete anything from that directory.

[dhsathiya]

Then, I am not sure what went wrong exactly.
But the easiest solution is, create a fresh VPS and re-creating the site from backup.

I have not faced this issue before but,
There are a couple of things to try if you want and know your way around Docker 🤞

Before doing anything, take a backup of VPS to be on safe side.

1. It is possible to override Dockerfile's default CMD with docker-compose command, update the docker-compose.yml file accordingly.
   https://docs.docker.com/compose/compose-file/compose-file-v3/#command https://stackoverflow.com/questions/63883149/docker-compose-command-doesnt-override-dockerfile-cmd

2. Override the CMD of Mariadb Dockerfile, build a new image with the --tc-heuristic-recover flag, update docker-compose.yml with your image.

[rishabhsdev]

Ok, is there a way to take a backup of my wp database using easyengine?
Once I have the backup of my posts, I can delete the site and create a new site and restore.

[dhsathiya]

The DB container is restarting. You won't be able to take a backup probably.
Best solution is to create a new VPS, install EE, recreate site.

A fresh VPS is needed as it seems like something went wrong with MariaDB while you did cleanup.

[freedog96150]

I would hazard a guess that looking at Docker logs should raise the reason why the container is constantly restarting. I have certainly been there with Docker many times in the EasyEngine environment and with some patience and sleuthing I have been able to resolve 100% of my issues without resorting to a complete rebuild from scratch.

But as mentioned above, with the container constantly restarting you will never be able to use any sort of normal database utility or WP plugin to get a backup. The Docker volume will persist and if you were to rebuilt a new container and just attach the same volume you should be able to gain access to the database container and all the stored data.

If this is still and issue and you have not moved on or resolved, let me know and we can do some more troubleshooting.