extract: Detect location of whosonfirst data from pelias.json #235
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This change allows the Placeholder extract script to work in most cases _without_ specifying the `WOF_DIR` environment variable. Previously, unless you were using the particular arrangement of files and directories from pelias/docker, the default location the extract script looks for data (`/data/whosonfirst-data/sqlite`) was probably not correct. I noticed this inconvenience when running Pelias locally _without_ docker for the first time in quite a long time. My guess/recollection is an older version of the extract script (pre-sqlite) was pure bash, and so checking `pelias.json` was less convenient than in the current Node.js script. The `WOF_DIR` environment variable is left as an override, but my hope is with this change almost no one would have to use it.
missinglink
reviewed
Feb 13, 2025
| @@ -9,7 +9,9 @@ const combinedStream = require('combined-stream'); | |||
|
|
|||
| const SQLITE_REGEX = /whosonfirst-data-[a-z0-9-]+\.db$/; | |||
|
|
|||
| const WOF_DIR = process.env.WOF_DIR || '/data/whosonfirst-data/sqlite'; | |||
| const WOF_DIR = process.env.WOF_DIR || // TODO: deprecate WOF_DIR env var after some time | |||
| config.whosonfirst.datapath || // Preferred method of finding WOF data is to autodetect from pelias.json | |||
There was a problem hiding this comment.
With ES6 this can be written as config?.whosonfirst?.datapath which will avoid fatal errors if the parent path doesn't exist.
Likely not an issue in practise as we have a default value
missinglink
approved these changes
Feb 13, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This change allows the Placeholder extract script to work in most cases without specifying the
WOF_DIRenvironment variable.Previously, unless you were using the particular arrangement of files and directories from pelias/docker, the default location the extract script looks for data (
/data/whosonfirst-data/sqlite) was probably not correct.I noticed this inconvenience when running Pelias locally without docker for the first time in quite a long time.
My guess/recollection is an older version of the extract script (pre-sqlite) was pure bash, and so checking
pelias.jsonwas less convenient than in the current Node.js script.The
WOF_DIRenvironment variable is left as an override, but my hope is with this change almost no one would have to use it.