Silent failure when Loading Quads (Out of memory killed?) #131
Comments
|
@C-Loftus The batch size is much to small, can you please try again with 1000000 (one million). QLever creates one partial vocabulary per batch. In the merging stage (where your index build) crashes, these partial vocabularies are merged. On some systems this crashes if there are too many partial vocabularies. There is no need to make a batch very small. A single batch has to fit into RAM, that's all. So one million triples per batch should never be a problem. |
|
@hannahbast Thank you for your response. Unfortunately that doesnt seem to work for me. I changed [data]
NAME = geoconnex
GET_DATA_CMD = less iow-dump.nq
DESCRIPTION = geoconnex
[index]
INPUT_FILES = iow-dump.nq
# INPUT_FILES = small_iow.nq
CAT_INPUT_FILES = cat ${INPUT_FILES}
SETTINGS_JSON = { "ascii-prefixes-only": false, "num-triples-per-batch": 1000000 }
[server]
PORT = 8888
ACCESS_TOKEN = _IbQrZYQE4TEX
[runtime]
SYSTEM = docker
IMAGE = docker.io/adfreiburg/qlever:latest
[ui]
UI_PORT = 8176
UI_CONFIG = default
|
|
@C-Loftus Can you paste the index log? And can you provide a link to your input file? The server log you posted indicates that the index build did not complete. |
|
@hannahbast Thank you for your reply. Here is the full index log. I briefly caught the memory in the container for the indexing go up to 7.4ish GB (the max) before it crashed for what its worth. The container is killed and removed and I can no longer see it in Docker desktop after this occurs. The link to the data can be downloaded here: https://zenodo.org/records/14853116 If zenodo does not work for any reason, please let me know and I find an alternative way to send the data. Index log: (note that indexing exits 0 and the container is not running, so doesn't appear there are any other indexing processes in the background) server log |
|
@C-Loftus Thanks for the link, I could build an index with the data without problems. Your machine or your Docker container seems to have little memory, but the dataset contains some very long lines (the longest line has almost a million characters). Can you try again with |
|
Hi Hannah, thank you for your help. I increased the RAM from 8gb to 20gb in Docker desktop and it seems that worked. It appears 8gb was not enough to ingest the data. I appreciate your help |
|
@C-Loftus Thanks for the feedback and happy to hear that it worked. The problem with limited memory is that the operating system can decide to kill the process and then the process just terminates and there is no opportunity to output a proper error message. It's not a QLever-specific problem. |
Background
I am trying to load a 8gb nquad file with qlever. The container exits without any error logs, but then when I run
qlever startit does not work. Either because the vocabulary ormeta-data.jsonfiles aren't present. Something appears to have silently failed generating the indexHowever, the index operation succeeds if I trim down my nquad file size to the first 100k lines or so. These lines were picked by random and I don't think there are any encoding or character issues in the rest of the file (it is a direct export from a valid graphdb). It seems to work simply by reducing the size.
I am assuming there is an OOM error when it reads/processes the file before the data is batched but that is just a guess.
What I have tried
I have tried changing
"num-triples-per-batch": 500anywhere from 500 to 10000 but didn't see much difference.I have tried giving it lots of memory via the cli with
--stxxl-memory 20GMachine / Versions
Machine: M3 Macbook Pro; Sonoma 14.3; 36 GB RAM
Installed qlever controller using
pipx 1.7.1Installed
qlever 0.5.17Other Issues
Seems to be similar to #113 and #111 and #73 I wanted to file this since I am using quads and I don't think others were.
Logs
Click to expand Logs
Docker Stats Graphs right before container exits
The text was updated successfully, but these errors were encountered: