Cannot use custom fields in Items

[giorgiobasile]

Hi, I'm trying to upload items adhering with STAC Rendering extension using the POST /collection/{collection_id}/items API.

The extension mandates the specification of a renders object for each Item as a custom key - not as a property - with information about the item rendering. While pystac is able to validate it and produce a well-formed JSON, the API seems to ignore the additional renders object.

The body I'm sending looks like:

{
    "type": "Feature",
    "stac_version": "1.0.0",
    "id": "nexgddp-storm-global-2080-rcp85",
    "properties": {
        "proj:epsg": 3857,
        "datetime": "2080-01-01T00:00:00Z"
    },
    "geometry": {
      ...
    },
    "links": [
       ...
    ],
    "assets": {
      ...
    },
    "bbox": [
        -180.0,
        -85.0511287798066,
        180.0,
        85.05112877980659
    ],
    "stac_extensions": [
        "https://stac-extensions.github.io/render/v1.0.0/schema.json"
    ],
    "renders": {
        "webmap": {
            "title": "Storm risk (2080, RCP8.5)",
            "colormap_name": "blues",
            "return_mask": true,
        }
    },
}

I get back 200 OK, and the same JSON but without renders.

I then tried adding other custom keys, unrelated to any extension - i.e. {'test': 'test'} but still got the same behavior. This looks to me like a bug, as the readme says:

PgSTAC stores all collection and item records as jsonb fields exactly as they come in allowing for any custom fields to be stored and retrieved transparently.

Are you able to reproduce it? Maybe there are other reasons why the endpoint behaves this way? Also, not sure whether this is actually regulated by downstream projects like stac-fastapi and pgstac.

Thanks in advance for any help!

[giorgiobasile]

thanks a lot @jonhealy1 for pointing me to this!

I think it is sensible that there is no explicit "render" field, as it would be impossible to support all possible additional fields - whether coming from an extension or not.
At the same time, pydantic allows for extra fields to be kept in the model instance; so that they would survive serialization and de-serialization. I'm not sure whether using that configuration would be enough - and what side effects would bring. Any thoughts? Thanks again.

[jonhealy1]

Hi I erased my comment because I was second-guessing myself but I was thinking that the issue arises here

[jonhealy1]

@giorgiobasile Hi. Is your field being saved in Postgres after you add an item? I'm wondering if the field makes it to the database or if stac-fastapi is preventing it from even getting there? The field could be disappearing before it gets to the database or afterwards when the api returns a stac-fastapi type Item.

[giorgiobasile]

@giorgiobasile Hi. Is your field being saved in Postgres after you add an item? I'm wondering if the field makes it to the database or if stac-fastapi is preventing it from even getting there? The field could be disappearing before it gets to the database or afterwards when the api returns a stac-fastapi type Item.

Good point, I can confirm that the item in the DB doesn't have the renders object. So, the field is disappearing before it gets to the database.
I think I can also confirm that the pydantic model you shared before is the one used by stac-fastapi transactional endpoints to parse Items - see here. Then, it makes sense that custom fields are immediately lost.

[jonhealy1]

Maybe we should move this issue over to the stac-fastapi parent repository as I think it would be an issue in other backends like stac-fastapi-elasticsearch as well?

[giorgiobasile]

Yes, I think all backends would be affected. Opened stac-utils/stac-fastapi#646, thanks again for helping out!