Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to download more than one .osm.pbf? #367

Open
gztchan opened this issue Feb 19, 2023 · 3 comments · May be fixed by #436
Open

How to download more than one .osm.pbf? #367

gztchan opened this issue Feb 19, 2023 · 3 comments · May be fixed by #436

Comments

@gztchan
Copy link

gztchan commented Feb 19, 2023

docker run \
    -e DOWNLOAD_PBF=https://download.geofabrik.de/europe/luxembourg-latest.osm.pbf \
    -e DOWNLOAD_POLY=https://download.geofabrik.de/europe/luxembourg.poly \
    -v osm-data:/data/database/ \
    overv/openstreetmap-tile-server \
    import

If I rerun the above command, and try to download another .osm.pdf, it shows that createuser: error: creation of new role failed: ERROR: role "renderer" already exists

I checked out the file and found that import always creates a new renderer user in Postgres

@gztchan gztchan changed the title How to How to download multiple .osm.pbf? Feb 19, 2023
@gztchan gztchan changed the title How to download multiple .osm.pbf? How to download more than 1 .osm.pbf? Feb 19, 2023
@gztchan gztchan changed the title How to download more than 1 .osm.pbf? How to download more than one .osm.pbf? Feb 19, 2023
@Istador
Copy link
Contributor

Istador commented Feb 22, 2023

import is only expected to be run once, because even if it would continue past the role "renderer" already exists error, it would not "add" to the existing data, but replace everything that is already imported.

You need to download and merge the files for multiple regions locally before importing them, or call osm2pgsql with append manually (SLOW).

@VicBancroft3D
Copy link

VicBancroft3D commented Jan 22, 2024

We have a similar issue in our usage of the tile server. In our use case, we would like to import additional regions and have it load it into the prior postgis database. In this case, the database has already been initialized with the renderer role and the gis database has been created and extended with both postgis and hstore . . .

One solution is to introduce an environment variable, SKIP_DB_INIT, such that if set, skips the initialization procedures.
For example, the relevant snippet from run.sh would be as follows.

    if [ -z ${SKIP_DB_INIT} ] ; then
        sudo -u postgres createuser renderer
        sudo -u postgres createdb -E UTF8 -O renderer gis
        sudo -u postgres psql -d gis -c "CREATE EXTENSION postgis;"
        sudo -u postgres psql -d gis -c "CREATE EXTENSION hstore;"
        sudo -u postgres psql -d gis -c "ALTER TABLE geometry_columns OWNER TO renderer;"
        sudo -u postgres psql -d gis -c "ALTER TABLE spatial_ref_sys OWNER TO renderer;"
        setPostgresPassword
    else
        echo "SKIP_DB_INIT is set."
    fi 

The environment variable could be set as follows.

% docker run -e SKIP_DB_INIT=true \
   -v /opt/assets/osm/north-carolina-latest.osm.pbf:/data/region.osm.pbf \
   -v osm-data:/data/database/ \
   overv/openstreetmap-tile-server:debug \
   import

Is there a down side to this ?

@Istador
Copy link
Contributor

Istador commented Jan 23, 2024

Is there a down side to this ?

Yes,

it would not "add" to the existing data, but replace everything that is already imported.

For this to work you'd also need to change the osm2pgsql command to append, which would run very slow compared to a import.


The recommended solution for multiple regions is to

download and merge the files for multiple regions locally before importing them

@hutli hutli linked a pull request Oct 9, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants