Data not syncing #68
Comments
|
Make sure you do this |
|
I followed the tutorial and am also having issues with the data not syncing... the service seems to start up and then do nothing. I tried running |
|
Make sure that the indexes you specified already exist in the database and in meillisearch |
|
Both the table I'm querying does exist, and AFAIK so does the index (confirmed via the Meilisearch REST API) |
|
Can you share the config file with me? |
|
I got it working by using this docker image |
|
I am using the official image but the problem seems to be in your database configuration, check the ports |
|
Which Ports do I need to check? |
|
Try that command |
|
Do not forget to create the tables that you specified in the configuration file |
|
Hey, I have the same error, I don't use docker. I have a local Postgres 15 instance on Debian 12. The error pattern looks exactly as described in the issue. |
|
Brief feedback from me, the problem has been solved. I started the sync again today and it worked. Unfortunately, I can't say what caused it, maybe I forgot to restart some service of the database. |
|
I have the same issue as @MartijnStraatman . When I run Meilisync I have the following log : It look promising and in my database log I have : However, despite those messages that seem to indacte that Meilisync and my database have established some kind of communication, nothing shows up in the Meili Search index. Here is my Meili Sync configuration : I append that to my Postgresql configuration : And I checked in the file system of my docker container to make sure that I haven't messed up and that my custom config file is used. I have no idea why my database is not syncing with Meili Search. If you have any ideas I'd love to hear them. |
|
I'm having the same issue on Azure Postgresql Flexible Server apiVersion: v1
kind: ConfigMap
metadata:
name: meilisync-config
namespace: vertex-dev
data:
config-template.yml: |
debug: true
meilisearch:
api_url: ${API_URL}
api_key: ${API_KEY}
progress:
type: file
source:
type: postgres
host: redatcted
port: 5432
database: redatcted
user: meilisync
password: ${PG_PASS}
sync:
- table: public."WasteCategory"
index: waste_category
pk: id
full: true
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: meilisync-deployment
namespace: vertex-dev
spec:
replicas: 1
selector:
matchLabels:
app: meilisync
template:
metadata:
labels:
app: meilisync
spec:
containers:
- name: meilisync
image: long2ice/meilisync:sha-f837e60
volumeMounts:
- name: config
mountPath: /meilisync/config.yml
subPath: config.yml
initContainers:
- name: config-init
image: alpine
command: ['sh', '-c', 'apk add --no-cache gettext && envsubst < /config-template/config-template.yml > /config/config.yml && cat /config/config.yml']
env:
- name: API_URL
valueFrom:
secretKeyRef:
name: meilisync-secret
key: api_url
- name: API_KEY
valueFrom:
secretKeyRef:
name: meilisync-secret
key: api_key
- name: PG_PASS
valueFrom:
secretKeyRef:
name: meilisync-secret
key: pg_pass
volumeMounts:
- name: config-template
mountPath: /config-template
- name: config
mountPath: /config
volumes:
- name: config-template
configMap:
name: meilisync-config
- name: config
emptyDir: {} |
|
Ok I manage to find the issue. In my case, meilisync managed to launch the syn task on my meilisearch instance. However, due to an error in the data, the tasks all failed. The error passed silently as meilisync does not check if the tasks finish successfully before login that the sync was a success. It would be good for the user experience to have some kinds of feedback about the status of the task launched by meilisync or to have this behavior clearly stated in the doc. |
|
Thanks for the update @Jefidev
2024-08-07 15:10:12.279 | INFO | meilisync.main:_:93 - Full data sync for table "redacted.public."redacted"" done! 35 documents added.
2024-08-07 15:10:12.279 | INFO | meilisync.main:_:101 - Start increment sync data from "SourceType.postgres" to MeiliSearch...
2024-08-07 15:10:12.286 | DEBUG | meilisync.main:_:104 - progress={'start_lsn': '97/CD000568'} |
Data is not syncing between postgres:16-bookworm ( with wal2json installed) and meilisync.
Logs:
Logs PSQL:
Some help is appreciated!
The text was updated successfully, but these errors were encountered: