OpenHIM arbitrarily drops MongoDB Collections

[moshonk]

We are experiencing an extremely peculiar issue with OpenHIM that is closely related to this issue #220. Collections keep getting dropped from the mongodb

We have experienced it severally, but not found a way around it. Here are the details:

1. All services seem to be running well.

2. An attempted login generates the following error:

openhim-core | 2023-02-27T05:38:05.566Z [worker1] info: Could not find user by email [email protected]

3. Accessing the Mongo database.

docker exec -it openhim-mongo mongo -host mongodb://mongo/openhim

4. This only displays 3 collections.

db.getCollectionNames()
[ "auditMeta", "audits", "log" ]

Software versions:

Ubuntu Version - 20.04.
Docker Version - 20.10.17
OpenHIM Version - 7.0.x
Mongo Version - v4.0.

Here's the docker-compose file that we are using: https://github.com/IntelliSOFT-Consulting/LHSS-CBDHS-HIE/blob/main/docker-compose.yml

Has anyone experienced this issue again? What was the workaround?

@rcrichton
@BMartinos

[rcrichton]

Hmm, I haven't experienced this before. My first thought is maybe it's something that is happening to the mongo container. There is nothing that removes the users collection in the OpenHIM. Here are a few questions to help us narrow it down:

• Has the mongo container been restarting? Maybe check the logs for anything out of place.
• How is the disk space on the server? Could mongo be running out of space?
• Do you have anything that cleans up docker volumes? Maybe the volume is getting removed?

[brianraila]

@rcrichton
I do have enough space on the server.
Additionally, I haven't noticed the restarts and nothing cleans up docker volumes.
However, I will continue to monitor that.

[rcrichton]

Ok, please let me know if you continue to experience this. I can't think of anything else that might be causing this.

[brianraila]

Hi @rcrichton we did manage to resolve this.

The issue was the Mongo port is mapped externally and some bot application was dropping the collections.

Removing the mapping resolves this issue.