List of all available options to use with planetiler?

[flomickl]

I currently use planetiler via docker/podman, as mentioned in the readme [1].
In this example, some additional arguments are mentioned, like

--download --area=monaco --force

In some other examples where planetiler is used and in the release notes [2] I find more, but either I don't know what it means or what its purpose is.
Is there a list of all the available options I can use with the Docker or Jar deployment to configure the output and fine-tune planetiler?

[1] https://github.com/onthegomap/planetiler/blob/main/README.md#usage
[2] https://github.com/onthegomap/planetiler/releases

[msbarry]

When planetiler starts it prints out lines like:

0:00:00 DEB - argument: boundary_country_names=true (boundary layer: add left/right codes of neighboring countries)
0:00:00 DEB - argument: boundary_osm_only=false (boundary layer: only use OSM, even at low zoom levels)
0:00:00 DEB - argument: transportation_z13_paths=false (transportation(_name) layer: show all paths on z13)
0:00:00 DEB - argument: building_merge_z13=true (building layer: merge nearby buildings at z13)

Those are all of the command-line arguments you can change, for example --boundary-country-names=false --boundary-osm-only=true. You can run with --help to just print those and exit. You can also set them via environmental variables like PLANETILER_BOUNDARY_COUNTRY_NAMES=false PLANETILER_BOUNDARY_OSM_ONLY=true.

[flomickl]

ah thanks for the hint.
Do you know if it is possible to pass some arguments for the output?
Instead of mbtiles I want to have *.pbf in the folder structure (z, x, y)

[msbarry]

There isn't a native option for writing to files yet (#536), but you can use the external tile-writer plugin framework @bbilger added in #639 to do it. Look at the s3 example, it should be pretty straightforward to replace s3-upload with writing to a local file on disk

[bbilger]

something like this should do the trick

create a simple script like this

#!/bin/bash
MAP_OUT_DIR=./map_out
while IFS="," read -r x y z encoded
do
  mkdir -p "$MAP_OUT_DIR/$z/$x"
  echo $encoded | base64 -d > "$MAP_OUT_DIR/$z/$x/$y.pbf"
done < "${1:-/dev/stdin}"

and then run

mkfifo /tmp/output.csv
./dir-out < /tmp/output.csv &
java -Xmx1g -jar planetiler.jar --tile_compression=none --download --area=monaco --append --output=/tmp/output.csv

if you want compressed tiles use --tile_compression=gzip

[flomickl]

Hi @bbilger,

Thanks for the example. I also took a look on #639
but I have a couple of questions due I was not able to finish the process.

Firstly, you mentioned the script part. Does this script have a specific name? How is it related to Planetiler command and how do they "find" each other? I but it next to my planetiler.jar simple as script.sh.

Maybe related to my first question, I get a broken pipe message after executing the run command.

mkfifo  /some/long/path/to/location/tmp/output.csv
./dir-out <  /some/long/path/to/location/tmp/output.csv &
java -Xmx1g -jar planetiler.jar --tile_compression=none --download --area=monaco --append --output= /some/long/path/to/location/tmp/output.csv

...
0:03:06 INF [archive] - Starting...
0:03:08 INF [archive:write] - Starting z0
0:03:08 INF [archive:write] - Finished z0 in 0s cpu:0s avg:11,5, now starting z1
0:03:08 INF [archive:write] - Finished z1 in 0s cpu:0s avg:0, now starting z2
Worker archive_write-1 died
java.io.UncheckedIOException: java.io.IOException: Data transmission interrupted (broken pipe)
        at com.onthegomap.planetiler.stream.WriteableCsvArchive$CsvTileWriter.write(WriteableCsvArchive.java:140)
        at com.onthegomap.planetiler.archive.TileArchiveWriter.tileWriter(TileArchiveWriter.java:360)
        at com.onthegomap.planetiler.worker.WorkerPipeline$Builder.lambda$sinkTo$2(WorkerPipeline.java:275)
        at com.onthegomap.planetiler.worker.Worker.lambda$new$0(Worker.java:41)
        at com.onthegomap.planetiler.worker.Worker.lambda$new$1(Worker.java:68)
        at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)
        Suppressed: java.io.UncheckedIOException: java.io.IOException: Datenübergabe unterbrochen (broken pipe)
                at com.onthegomap.planetiler.stream.WriteableCsvArchive$CsvTileWriter.close(WriteableCsvArchive.java:149)
                at com.onthegomap.planetiler.archive.TileArchiveWriter.tileWriter(TileArchiveWriter.java:342)
                ... 7 more
        Caused by: java.io.IOException: Data transmission interrupted (broken pipe)
                at java.base/sun.nio.ch.UnixFileDispatcherImpl.write0(Native Method)
                at java.base/sun.nio.ch.UnixFileDispatcherImpl.write(UnixFileDispatcherImpl.java:65)
                at java.base/sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:137)
                at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:102)
                at java.base/sun.nio.ch.IOUtil.write(IOUtil.java:72)
                at java.base/sun.nio.ch.FileChannelImpl.write(FileChannelImpl.java:300)
                at java.base/sun.nio.ch.ChannelOutputStream.writeFully(ChannelOutputStream.java:68)
                at java.base/sun.nio.ch.ChannelOutputStream.write(ChannelOutputStream.java:105)
                at org.apache.logging.log4j.core.util.CloseShieldOutputStream.write(CloseShieldOutputStream.java:53)
                at java.base/sun.nio.cs.StreamEncoder.writeBytes(StreamEncoder.java:309)
                at java.base/sun.nio.cs.StreamEncoder.implClose(StreamEncoder.java:432)
                at java.base/sun.nio.cs.StreamEncoder.lockedClose(StreamEncoder.java:237)
                at java.base/sun.nio.cs.StreamEncoder.close(StreamEncoder.java:222)
                at java.base/java.io.OutputStreamWriter.close(OutputStreamWriter.java:266)
                at java.base/java.io.BufferedWriter.implClose(BufferedWriter.java:398)
                at java.base/java.io.BufferedWriter.close(BufferedWriter.java:380)
                at com.onthegomap.planetiler.stream.WriteableCsvArchive$CsvTileWriter.close(WriteableCsvArchive.java:147)
                ... 8 more
...

[1]+  Exit 127                ./dir-out < /some/long/path/to/location/tmp/output.csv

Have I done something wrong?

[bbilger]

@flomickl
The name of the script in the example is dir-out and it's located in the current directory. If your script is in a different location or named differently then you need to adjust
./dir-out < /tmp/output.csv & to /path/to/script.sh < /tmp/output.csv &

To answer your other question. Planetiler does not (need to) know anything about the script and as such does not need to find it. Planetiler simply writes to a file (/tmp/output.csv). The trick now is that the file is not an ordinary file but a named pipe (fifo). All data planetiler writes to that file is further piped into your script (./dir-out < /tmp/output.csv & ) to process it (line by line in that case).

Maybe one more note here in general: writing to a named pipe is blocking until someone reads from it thus we need to setup the script that reads from the pipe before starting planetiler, and run it in the background (&) since it blocks until it it's done - but you could also run that in a different terminal or so.