This tool is designed to synchronize data between a Postgresql database and Elasticsearch. It uses a trigger system with a pub/sub mechanism to monitor real-time changes in the database and index them in Elasticsearch.
- Pull the Docker image from the GitHub Container Registry:
docker pull ghcr.io/quix-labs/pg-el-sync:latest
- Install the tool using Go:
go install github.com/quix-labs/pg-el-sync@latest
You can also install the tool using release assets such as .deb
, .apk
, or others.
Download the appropriate package from the Releases page, and then follow the instructions provided for your specific platform.
The tool provides two main commands for usage:
pg-el-sync listen
: Start listening to the PostgreSQL database for real-time changes and sync them with Elasticsearch.pg-el-sync index
: Index all tables from the PostgreSQL database into Elasticsearch.
docker run -v /path/to/config.yaml:/app/config.yaml ghcr.io/quix-labs/pg-el-sync:latest pg-el-sync listen
docker run --rm -v /path/to/config.yaml:/app/config.yaml ghcr.io/quix-labs/pg-el-sync:latest pg-el-sync index
By default pg-el-sync
load configuration file from /app/config.yaml
, you can override using environment variable:
export CONFIG_FILE=/path/to/config.yaml
# Or with docker: docker run -e CONFIG_FILE=/path/in/docker/config.yaml ...
To manage the pg-el-sync listen
command with Supervisord, you can use the following configuration:
[program:pg-el-sync]
command=/path/to/pg-el-sync listen
autostart=true
autorestart=true
startsecs=10
startretries=3
stdout_logfile=/var/log/pg-el-sync-listen.log
stderr_logfile=/var/log/pg-el-sync-listen.err.log
Make sure to replace /path/to/pg-el-sync
with the absolute path to your pg-el-sync
installation.
You can also adjust other settings according to your needs, such as log file paths and startup parameters.
To set up the environment for local development, follow these steps:
-
Clone this repository:
git clone https://github.com/quix-labs/pg-el-sync
-
Navigate to the
pg-el-sync
directory:cd pg-el-sync
-
Copy the
config.example.yaml
file toconfig.yaml
and configure it according to your environment:cp config.example.yaml config.yaml
-
Run the tool in development mode:
go run .
This project is licensed under the MIT License.