Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[tile-join with PMTiles files] Possible race condition? Different errors every time, sometimes completes successfully #278

Open
iam-spartacus opened this issue Oct 6, 2024 · 1 comment

Comments

@iam-spartacus
Copy link

iam-spartacus commented Oct 6, 2024

I'm having issues joining PMTiles files with the 'tile-join' program. I get different errors every time, and sometimes the program will complete successfully. I've tested on Debian 12 and Ubuntu 24.04, and both OSes give me a wide variety of errors.

Here's some example code that I've been using for my tests.

First, using the 'go-pmtiles' program, I extract a couple of PMTiles files (roughly 200-250MB each) from the Protomaps website:

#!/bin/bash

# using go-pmtiles to download a couple of small PMTiles sections
# from https://maps.protomaps.com/builds/
./pmtiles extract https://build.protomaps.com/20240617.pmtiles debug_quebec.pmtiles --bbox=-74.134369,45.264255,-70.955200,47.007416
./pmtiles extract https://build.protomaps.com/20240617.pmtiles debug_maritimes.pmtiles --bbox=-67.478027,43.181147,-61.545410,47.100045

After that, I run the 'tile-join' program. I'm using a script to run batches of 10 trials, because I get different results every time I run it. Additionally, some trials will completely freeze my system, and I have to restart my computer.

#!/bin/bash

for i in {1..10}
do
	echo Trial $i
	tile-join --force -pk -o merged_$i.pmtiles debug_*.pmtiles
done

My results from a sample run of 10 trials (Note: Trial 7 was successful).

Trial 1: Fatal glibc error: malloc.c:4192 (_int_malloc): assertion failed: chunk_main_arena (fwd)
Trial 2: Internal error: corrupted geometry
Trial 3: Decompression error: data error
Trial 4: free(): invalid size 6785 Aborted (core dumped)
Trial 5: 7076 Segmentation fault (core dumped)
Trial 6: Decompression error: data error

Trial 7: SUCCESS!!!

Trial 8: 12027 Segmentation fault (core dumped)
Trial 9: Internal error: corrupted geometry
Trial 10: Internal error: corrupted geometry

As you can see, every once in a while, 'tile-join' will complete successfully.

Given the wide variety of outcomes, I'm wondering if there is a race condition somewhere...

@e-n-f
Copy link
Collaborator

e-n-f commented Oct 7, 2024

Thanks for the report! I'll take a look and see if I can reproduce the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

2 participants