Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

replace searchid by search_id in path parameters and searchid by id in response model #129

Merged
merged 4 commits into from
Oct 12, 2023
Merged

replace searchid by search_id in path parameters and searchid by id in response model #129

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bbd4961
rename searchid path parameter to search_id
vincentsarago Oct 11, 2023
0e5d328
rename `searchid` to `id` in response model
vincentsarago Oct 12, 2023
7695e1d
update tests and docs
vincentsarago Oct 12, 2023
b2ea9ce
fix benchmark
vincentsarago Oct 12, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,25 @@
## Unrelease

* add `pgstac_dependency` attribute in `MosaicTilerFactory` (defaults to `dependencies.PgSTACParams`)

* add database's `pool` check in startup event

* rename `dependencies.PathParams` to `dependencies.SearchIdParams` **breaking change**

* rename `searchid` path parameter to `search_id` **breaking change**

* in `model.RegisterResponse` (model used in `/register` endpoint) rename `searchid` by `id` **breaking change**

```python
# before
resp = httpx.post("/mosaic/register", body={"collections": ["my-collection"], "filter-lang": "cql-json"})
assert resp.json()["searchid"]

# now
resp = httpx.post("/mosaic/register", body={"collections": ["my-collection"], "filter-lang": "cql-json"})
assert resp.json()["id"]
```

## 0.8.0 (2023-10-06)

* update titiler requirement to `>=0.14.0,<0.15`
Expand Down
30 changes: 15 additions & 15 deletions benchmark/stac/items.json
View file
Open in desktop

Large diffs are not rendered by default.

88 changes: 74 additions & 14 deletions docs/src/intro.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,22 +8,24 @@ By default the main application (`titiler.pgstac.main.app`) provides two sets of

- `/collections/{collection_id}/items/{item_id}`: Dynamic tiler for single STAC item (stored in PgSTAC)

## Mosaic
## Virtual Mosaic

The goal of the `mosaic` endpoints is to use any [`search`](https://github.com/radiantearth/stac-api-spec/tree/master/item-search) query to create tiles. `titiler-pgstac` provides a set of endpoint to `register` and `list` the `search` queries.

### Register a `Search` request
### Register a PgSTAC `Search` request

![](https://user-images.githubusercontent.com/10407788/132193537-0560016f-09bc-4a25-8a2a-eac9b50bc28a.png)

!!! Important
In `TiTiler.PgSTAC` a STAC [`Search Query`](https://github.com/radiantearth/stac-api-spec/tree/master/item-search) is equivalent to a Mosaic.
In `TiTiler.PgSTAC` a STAC [`Search Query`](https://github.com/radiantearth/stac-api-spec/tree/master/item-search) is equivalent to a *Virtual Mosaic*.

Before being able to create Map Tiles, the user needs to register a `Search Query` within the PgSTAC database (in the `searches` table). By default, `TiTiler.PgSTAC` has a `/mosaic/register (POST)` endpoint which will:

- validate the search query (based on the STAC API specification [`item-search`]((https://github.com/radiantearth/stac-api-spec/tree/master/item-search)))

- send the search query to the postgres database using the [`search_query`](https://github.com/stac-utils/pgstac/blob/76512ab50e1373e3f77c65843cf328cbe6dd0dec/sql/004_search.sql#L1000) PgSTAC function
- return a `searchid` (might be also called `mosaicid`).

- return a pgstac search identifier (`id`) (might be also called `mosaicid`).

**Example**

Expand All @@ -34,7 +36,7 @@ curl -X 'POST' 'http://127.0.0.1:8081/mosaic/register' \
-d '{"collections":["landsat-c2l2-sr"], "bbox":[-123.75,34.30714385628804,-118.125,38.82259097617712], "filter-lang": "cql-json"}' | jq

>> {
"searchid": "5a1b82d38d53a5d200273cbada886bd7",
"id": "5a1b82d38d53a5d200273cbada886bd7",
"links": [
{
"rel": "metadata",
Expand All @@ -56,7 +58,7 @@ curl -X 'POST' 'http://127.0.0.1:8081/mosaic/register' \
-d '{"filter": {"op": "and", "args": [{"op": "=", "args": [{"property": "collection"}, "landsat-c2l2-sr"]}, {"op": "s_intersects", "args": [{"property": "geometry"}, {"coordinates": [[[-123.75, 34.30714385628804], [-123.75, 38.82259097617712], [-118.125, 38.82259097617712], [-118.125, 34.30714385628804], [-123.75, 34.30714385628804]]], "type": "Polygon"}]}]}}' | jq

>> {
"searchid": "5063721f06957d6b2320326d82e90d1e",
"id": "5063721f06957d6b2320326d82e90d1e",
"links": [
{
"rel": "metadata",
Expand All @@ -79,7 +81,7 @@ curl http://127.0.0.1:8081/mosaic/5063721f06957d6b2320326d82e90d1e/info | jq

>> {
"search": {
"hash": "5063721f06957d6b2320326d82e90d1e", # <-- this is the search/mosaic ID
"hash": "5063721f06957d6b2320326d82e90d1e", # <-- this is the search/mosaic identifier
"search": { # <-- Summary of the search request
"filter": { # <-- this is CQL2 filter associated with the search
"op": "and",
Expand Down Expand Up @@ -163,7 +165,7 @@ curl -X 'POST' 'http://127.0.0.1:8081/mosaic/register' \
-d '{"collections":["landsat-c2l2-sr"], "bbox":[-123.75,34.30714385628804,-118.125,38.82259097617712], "filter-lang": "cql-json", "metadata": {"minzoom": 8, "maxzoom": 13, "assets": ["B04", "B03", "B02"], "defaults": {"true_color": {"assets": ["B04", "B03", "B02"], "color_formula": "Gamma RGB 3.5 Saturation 1.7 Sigmoidal RGB 15 0.35"}}}}' | jq

>> {
"searchid": "f31d7de8a5ddfa3a80b9a9dd06378db1",
"id": "f31d7de8a5ddfa3a80b9a9dd06378db1",
"links": [
{
"rel": "metadata",
Expand Down Expand Up @@ -203,16 +205,16 @@ curl http://127.0.0.1:8081/mosaic/f31d7de8a5ddfa3a80b9a9dd06378db1/info | jq '.s

### Fetch mosaic `Tiles`

When we have a `searchid` we can now call the dynamic tiler and ask for Map Tiles.
When we have an **id** we can call the dynamic tiler and ask for Map Tiles.

![](https://user-images.githubusercontent.com/10407788/132197899-e79b3118-313b-45e7-a431-5d3034984459.png)

**How it works**

On each `Tile` request, the tiler api is going to call the PgSTAC [`geometrysearch`](https://github.com/stac-utils/pgstac/blob/76512ab50e1373e3f77c65843cf328cbe6dd0dec/sql/006_tilesearch.sql#L4) function with the `searchid` and the Tile geometry to get the list of STAC items ([code](https://github.com/stac-utils/titiler-pgstac/blob/0f2b5b4ba50bb3458237ab21cf9a154d7b811851/titiler/pgstac/mosaic.py#L238-L247)). Then based on the `assets` parameter, the tiler will construct the tile image ([code](https://github.com/stac-utils/titiler-pgstac/blob/0f2b5b4ba50bb3458237ab21cf9a154d7b811851/titiler/pgstac/mosaic.py#L257-L263)).
On each `Tile` request, the tiler api is going to call the PgSTAC [`geometrysearch`](https://github.com/stac-utils/pgstac/blob/76512ab50e1373e3f77c65843cf328cbe6dd0dec/sql/006_tilesearch.sql#L4) function with the `id` and the `Tile geometry` to get the list of **STAC Items** ([code](https://github.com/stac-utils/titiler-pgstac/blob/0f2b5b4ba50bb3458237ab21cf9a154d7b811851/titiler/pgstac/mosaic.py#L238-L247)). Then based on the `assets` parameter, the tiler will construct the tile image ([code](https://github.com/stac-utils/titiler-pgstac/blob/0f2b5b4ba50bb3458237ab21cf9a154d7b811851/titiler/pgstac/mosaic.py#L257-L263)).

!!! important
Because `Tiles` will be created from STAC items we HAVE TO pass **`assets={stac asset}`** option to the tile endpoint.
Because `Tiles` will be created from **STAC Items** we HAVE TO pass **`assets={stac asset}`** option to the tile endpoint to tell the tiler which **STAC assets** has to be used.

See full list of [options](../mosaic_endpoints/#tiles)

Expand All @@ -222,14 +224,72 @@ On each `Tile` request, the tiler api is going to call the PgSTAC [`geometrysear
curl 'http://127.0.0.1:8081/mosaic/f1ed59f0a6ad91ed80ae79b7b52bc707/tiles/8/40/102.png?assets=B01&rescale=0,16000 > 8-40-102.png
```

## Items
## Individual Item

Set of endpoints created using TiTiler's [`MultiBaseTilerFactory`]() but with `item` and `collection` path parameter (instead of the `url=` query parameter).
`titiler-pgstac` can also be used to access individual item stored in the PgSTAC database. By default the `titiler-pgstac` application will have a set of `/collections/{collection_id}/items/{item_id}/...` endpoints. The endpoints are created using [titiler.core.factory.MultiBaseTilerFactory](https://developmentseed.org/titiler/advanced/tiler_factories/#titilercorefactorymultibasetilerfactory) but using a custom `path_dependency` with `collection_id` and `item_id` path parameters instead of the STAC url as query parameter.

**example**

```bash
curl http://127.0.0.1:8081/collections/landsat-c2l2-sr/items/LC08_L1TP_028004_20171002_20171018_01_A1
curl http://127.0.0.1:8081/collections/world/items/world_20000_5000/info | jq
{
"asset": {
"bounds": [
153.5000000000667,
-76.83333333336668,
179.8333333334053,
6.4999999999833165
],
"minzoom": 3,
"maxzoom": 6,
"band_metadata": [
[
"b1",
{}
],
[
"b2",
{}
],
[
"b3",
{}
]
],
"band_descriptions": [
[
"b1",
""
],
[
"b2",
""
],
[
"b3",
""
]
],
"dtype": "uint8",
"nodata_type": "None",
"colorinterp": [
"red",
"green",
"blue"
],
"driver": "GTiff",
"count": 3,
"width": 1580,
"height": 5000,
"overviews": [
2,
4,
8,
16
]
}
}

```

See full list of [endpoints](../item_endpoints)
56 changes: 28 additions & 28 deletions docs/src/mosaic_endpoints.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,17 +3,17 @@ The `titiler.pgstac` package comes with a full FastAPI application with Mosaic a
| Method | URL | Output | Description
| ------ | ---------------------------------------------------------------------------------|-----------------------------------------|--------------
| `POST` | `/mosaic/register` | JSON ([Register][register_model]) | Register **Search** query
| `GET` | `/mosaic/{searchid}/info` | JSON ([Info][info_model]) | Return **Search** query infos
| `GET` | `/mosaic/{search_id}/info` | JSON ([Info][info_model]) | Return **Search** query infos
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to have something similar as /collections/{collection_id}/items/{item_id}

| `GET` | `/mosaic/list` | JSON ([Infos][infos_model]) | Return list of **Search** entries with `Mosaic` type
| `GET` | `/mosaic/{searchid}/{lon},{lat}/assets` | JSON | Return a list of assets which overlap a given point
| `GET` | `/mosaic/{searchid}/tiles[/{TileMatrixSetId}]/{z}/{x}/{Y}/assets` | JSON | Return a list of assets which overlap a given tile
| `GET` | `/mosaic/{searchid}/tiles[/{TileMatrixSetId}]/{z}/{x}/{y}[@{scale}x][.{format}]` | image/bin | Create a web map tile image for a search query and a tile index
| `GET` | `/mosaic/{searchid}[/{TileMatrixSetId}]/tilejson.json` | JSON ([TileJSON][tilejson_model]) | Return a Mapbox TileJSON document
| `GET` | `/mosaic/{searchid}[/{TileMatrixSetId}]/WMTSCapabilities.xml` | XML | return OGC WMTS Get Capabilities
| `GET` | `/mosaic/{searchid}[/{TileMatrixSetId}]/map` | HTML | simple map viewer
| `POST` | `/mosaic/{searchid}/statistics` | GeoJSON ([Statistics][statitics_model]) | Return statistics for geojson features
| `GET` | `/mosaic/{searchid}/bbox/{minx},{miny},{maxx},{maxy}[/{width}x{height}].{format}`| image/bin | Create an image from part of a dataset
| `POST` | `/mosaic/{searchid}/feature[/{width}x{height}][.{format}]` | image/bin | Create an image from a GeoJSON feature
| `GET` | `/mosaic/{search_id}/{lon},{lat}/assets` | JSON | Return a list of assets which overlap a given point
| `GET` | `/mosaic/{search_id}/tiles[/{TileMatrixSetId}]/{z}/{x}/{Y}/assets` | JSON | Return a list of assets which overlap a given tile
| `GET` | `/mosaic/{search_id}/tiles[/{TileMatrixSetId}]/{z}/{x}/{y}[@{scale}x][.{format}]` | image/bin | Create a web map tile image for a search query and a tile index
| `GET` | `/mosaic/{search_id}[/{TileMatrixSetId}]/tilejson.json` | JSON ([TileJSON][tilejson_model]) | Return a Mapbox TileJSON document
| `GET` | `/mosaic/{search_id}[/{TileMatrixSetId}]/WMTSCapabilities.xml` | XML | return OGC WMTS Get Capabilities
| `GET` | `/mosaic/{search_id}[/{TileMatrixSetId}]/map` | HTML | simple map viewer
| `POST` | `/mosaic/{search_id}/statistics` | GeoJSON ([Statistics][statitics_model]) | Return statistics for geojson features
| `GET` | `/mosaic/{search_id}/bbox/{minx},{miny},{maxx},{maxy}[/{width}x{height}].{format}`| image/bin | Create an image from part of a dataset
| `POST` | `/mosaic/{search_id}/feature[/{width}x{height}][.{format}]` | image/bin | Create an image from a GeoJSON feature


### Register a Search Request
Expand Down Expand Up @@ -86,7 +86,7 @@ Example:
```bash
curl -X 'POST' 'http://127.0.0.1:8081/mosaic/register' -H 'accept: application/json' -H 'Content-Type: application/json' -d '{"collections":["landsat-c2l2-sr"], "bbox":[-123.75,34.30714385628804,-118.125,38.82259097617712], "filter-lang": "cql-json"}' | jq
>> {
"searchid": "5a1b82d38d53a5d200273cbada886bd7",
"id": "5a1b82d38d53a5d200273cbada886bd7",
"links": [
{
"rel": "metadata",
Expand All @@ -110,10 +110,10 @@ curl -X 'POST' 'http://127.0.0.1:8081/mosaic/register' -H 'accept: application/j

### Search infos

`:endpoint:/mosaic/{searchid}/info - [GET]`
`:endpoint:/mosaic/{search_id}/info - [GET]`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.

Example:

Expand Down Expand Up @@ -183,10 +183,10 @@ Example:

### Tiles

`:endpoint:/mosaic/{searchid}/tiles[/{TileMatrixSetId}]/{z}/{x}/{y}[@{scale}x][.{format}]`
`:endpoint:/mosaic/{search_id}/tiles[/{TileMatrixSetId}]/{z}/{x}/{y}[@{scale}x][.{format}]`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.
- **TileMatrixSetId**: TileMatrixSet name, default is `WebMercatorQuad`. OPTIONAL
- **z**: Tile's zoom level.
- **x**: Tile's column.
Expand Down Expand Up @@ -229,10 +229,10 @@ Example:

### TilesJSON

`:endpoint:/mosaic/{searchid}[/{TileMatrixSetId}]/tilejson.json`
`:endpoint:/mosaic/{search_id}[/{TileMatrixSetId}]/tilejson.json`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.
- **TileMatrixSetId**: TileMatrixSet name, default is `WebMercatorQuad`. OPTIONAL

- QueryParams:
Expand Down Expand Up @@ -273,10 +273,10 @@ Example:

### WMTS

`:endpoint:/mosaic/{searchid}[/{TileMatrixSetId}]/WMTSCapabilities.xml`
`:endpoint:/mosaic/{search_id}[/{TileMatrixSetId}]/WMTSCapabilities.xml`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.
- **TileMatrixSetId**: TileMatrixSet name, default is `WebMercatorQuad`. OPTIONAL

- QueryParams:
Expand All @@ -298,10 +298,10 @@ Example:

### Assets

`:endpoint:/mosaic/{searchid}/tiles/[{TileMatrixSetId}]/{z}/{x}/{y}/assets`
`:endpoint:/mosaic/{search_id}/tiles/[{TileMatrixSetId}]/{z}/{x}/{y}/assets`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.
- **TileMatrixSetId**: TileMatrixSet name, default is `WebMercatorQuad`. OPTIONAL
- **z**: Tile's zoom level.
- **x**: Tile's column.
Expand All @@ -318,10 +318,10 @@ Example:

- `https://myendpoint/mosaic/f1ed59f0a6ad91ed80ae79b7b52bc707/tiles/0/0/0/assets`

`:endpoint:/mosaic/{searchid}/{lon},{lat}/assets`
`:endpoint:/mosaic/{search_id}/{lon},{lat}/assets`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.
- **lon**: Longitude (in WGS84 CRS).
- **lat**: Latitude (in WGS84 CRS).

Expand All @@ -338,7 +338,7 @@ Example:

### Statistics

`:endpoint:/mosaic/{searchid}/statistics - [POST]`
`:endpoint:/mosaic/{search_id}/statistics - [POST]`

- Body:
- **feature** (JSON): A valid GeoJSON feature or FeatureCollection
Expand Down Expand Up @@ -379,12 +379,12 @@ Example:

### BBOX/Feature

`:endpoint:/mosaic/{searchid}/bbox/{minx},{miny},{maxx},{maxy}.{format}`
`:endpoint:/mosaic/{search_id}/bbox/{minx},{miny},{maxx},{maxy}.{format}`

`:endpoint:/mosaic/{searchid}/bbox/{minx},{miny},{maxx},{maxy}/{width}x{height}.{format}`
`:endpoint:/mosaic/{search_id}/bbox/{minx},{miny},{maxx},{maxy}/{width}x{height}.{format}`

- PathParams:
- **searchid**: search query hashkey.
- **search_id**: search query hashkey.
- **minx,miny,maxx,maxy** (str): Comma (',') delimited bounding box in WGS84.
- **format** (str): Output image format.
- **height** (int): Force output image height.
Expand Down Expand Up @@ -424,7 +424,7 @@ Example:
- `https://myendpoint/mosaic/f1ed59f0a6ad91ed80ae79b7b52bc707/bbox/0,0,10,10/400x300.png?assets=B01`


`:endpoint:/mosaic/{searchid}/feature[/{width}x{height}][].{format}] - [POST]`
`:endpoint:/mosaic/{search_id}/feature[/{width}x{height}][].{format}] - [POST]`

- Body:
- **feature** (JSON): A valid GeoJSON feature (Polygon or MultiPolygon)
Expand Down
4 changes: 2 additions & 2 deletions docs/src/notebooks/demo.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@
").json()\n",
"print(response)\n",
"\n",
"searchid = response[\"searchid\"]"
"searchid = response[\"id\"]"
]
},
{
Expand Down Expand Up @@ -495,7 +495,7 @@
").json()\n",
"print(response)\n",
"\n",
"searchid = response[\"searchid\"]"
"searchid = response[\"id\"]"
]
},
{
Expand Down
Loading