Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating to stac pydantic 3 #627

Closed
wants to merge 4 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/cicd.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ jobs:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.8", "3.9", "3.10", "3.11"]
python-version: ["3.9", "3.10", "3.11"]
timeout-minutes: 20

services:
Expand Down Expand Up @@ -48,7 +48,7 @@ jobs:
python-version: ${{ matrix.python-version }}

- name: Lint code
if: ${{ matrix.python-version == 3.8 }}
if: ${{ matrix.python-version == 3.9 }}
run: |
python -m pip install pre-commit
pre-commit run --all-files
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/deploy_mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@ jobs:
- name: Checkout main
uses: actions/checkout@v4

- name: Set up Python 3.8
- name: Set up Python 3.9
uses: actions/setup-python@v5
with:
python-version: 3.8
python-version: 3.9

- name: Install dependencies
run: |
Expand Down
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,6 @@ repos:
- id: ruff
args: [--fix, --exit-non-zero-on-fix]
- repo: https://github.com/psf/black
rev: 23.3.0
rev: 24.1.1
hooks:
- id: black
3 changes: 3 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,9 @@

## [Unreleased]

* Removing support for Python 3.8
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this needed? pydantic 2 does support python 3.8 https://github.com/pydantic/pydantic/blob/main/pyproject.toml#L65

* Update to pydantic v2 and stac_pydantic v3

## [2.4.9] - 2023-11-17

### Added
Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ $ cd stac-fastapi
$ pip install -e stac_fastapi/api[dev]
```

**Python3.8 only**
**Python3.9 only**

This repo is set to use `pre-commit` to run *ruff*, *pydocstring*, *black* ("uncompromising Python code formatter") and mypy when committing new code.

Expand Down
2 changes: 1 addition & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.8-slim as base
FROM python:3.9-slim as base

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agree to remove python 3.8 but then we should use python 3.11 IMO

# Any python libraries that require system libraries to be installed will likely
# need the following packages in order to build
Expand Down
2 changes: 1 addition & 1 deletion Dockerfile.docs
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
FROM python:3.8-slim
FROM python:3.9-slim

# build-essential is required to build a wheel for ciso8601
RUN apt update && apt install -y build-essential
Expand Down
1 change: 0 additions & 1 deletion mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@ nav:
- requests: api/stac_fastapi/types/requests.md
- rfc3339: api/stac_fastapi/types/rfc3339.md
- search: api/stac_fastapi/types/search.md
- stac: api/stac_fastapi/types/stac.md
- version: api/stac_fastapi/types/version.md
- Development - Contributing: "contributing.md"
- Release Notes: "release-notes.md"
Expand Down
7 changes: 2 additions & 5 deletions stac_fastapi/api/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@
desc = f.read()

install_requires = [
"attrs",
"pydantic[dotenv]<2",
"stac_pydantic==2.0.*",
"brotli_asgi",
"stac-fastapi.types",
]
Expand All @@ -32,12 +29,12 @@
description="An implementation of STAC API based on the FastAPI framework.",
long_description=desc,
long_description_content_type="text/markdown",
python_requires=">=3.8",
python_requires=">=3.9",
classifiers=[
"Intended Audience :: Developers",
"Intended Audience :: Information Technology",
"Intended Audience :: Science/Research",
"Programming Language :: Python :: 3.8",
"Programming Language :: Python :: 3.9",
"License :: OSI Approved :: MIT License",
],
keywords="STAC FastAPI COG",
Expand Down
59 changes: 32 additions & 27 deletions stac_fastapi/api/stac_fastapi/api/app.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
"""Fastapi app creation."""
from typing import Any, Dict, List, Optional, Tuple, Type, Union

from typing import Any, Optional, Tuple, Type, Union

import attr
from brotli_asgi import BrotliMiddleware
Expand All @@ -9,7 +10,7 @@
from stac_pydantic import Collection, Item, ItemCollection
from stac_pydantic.api import ConformanceClasses, LandingPage
from stac_pydantic.api.collections import Collections
from stac_pydantic.version import STAC_VERSION
from stac_pydantic.api.version import STAC_API_VERSION
from starlette.responses import JSONResponse, Response

from stac_fastapi.api.errors import DEFAULT_STATUS_CODES, add_exception_handlers
Expand Down Expand Up @@ -67,8 +68,8 @@ class StacApi:

settings: ApiSettings = attr.ib()
client: Union[AsyncBaseCoreClient, BaseCoreClient] = attr.ib()
extensions: List[ApiExtension] = attr.ib(default=attr.Factory(list))
exceptions: Dict[Type[Exception], int] = attr.ib(
extensions: list[ApiExtension] = attr.ib(default=attr.Factory(list))
exceptions: dict[Type[Exception], int] = attr.ib(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI is using dict directly instead of typing.Dict is the reason to switch to python>=3.9, typing.Dict is just an alias of dict (https://github.com/python/cpython/blob/3.9/Lib/typing.py#L1743C15-L1743C19) so it's fine to use typing.Dict IMO

https://stackoverflow.com/questions/37087457/difference-between-defining-typing-dict-and-dict

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same goes for List

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vincentsarago I found that during the create_request_model in models.py. The field_info.annotation would leave out the Optional part of the type. The only solution I found was to switch from typing List and Dict to the generic list and dict. There maybe a better solution to this?

default=attr.Factory(lambda: DEFAULT_STATUS_CODES)
)
app: FastAPI = attr.ib(
Expand All @@ -85,7 +86,7 @@ class StacApi:
router: APIRouter = attr.ib(default=attr.Factory(APIRouter))
title: str = attr.ib(default="stac-fastapi")
api_version: str = attr.ib(default="0.1")
stac_version: str = attr.ib(default=STAC_VERSION)
stac_version: str = attr.ib(default=STAC_API_VERSION)
description: str = attr.ib(default="stac-fastapi")
search_get_request_model: Type[BaseSearchGetRequest] = attr.ib(
default=BaseSearchGetRequest
Expand All @@ -95,12 +96,12 @@ class StacApi:
)
pagination_extension = attr.ib(default=TokenPaginationExtension)
response_class: Type[Response] = attr.ib(default=JSONResponse)
middlewares: List = attr.ib(
middlewares: list = attr.ib(
default=attr.Factory(
lambda: [BrotliMiddleware, CORSMiddleware, ProxyHeaderMiddleware]
)
)
route_dependencies: List[Tuple[List[Scope], List[Depends]]] = attr.ib(default=[])
route_dependencies: list[Tuple[list[Scope], list[Depends]]] = attr.ib(default=[])

def get_extension(self, extension: Type[ApiExtension]) -> Optional[ApiExtension]:
"""Get an extension.
Expand All @@ -125,9 +126,9 @@ def register_landing_page(self):
self.router.add_api_route(
name="Landing Page",
path="/",
response_model=LandingPage
if self.settings.enable_response_models
else None,
response_model=(
LandingPage if self.settings.enable_response_models else None
),
response_class=self.response_class,
response_model_exclude_unset=False,
response_model_exclude_none=True,
Expand All @@ -146,9 +147,9 @@ def register_conformance_classes(self):
self.router.add_api_route(
name="Conformance Classes",
path="/conformance",
response_model=ConformanceClasses
if self.settings.enable_response_models
else None,
response_model=(
ConformanceClasses if self.settings.enable_response_models else None
),
response_class=self.response_class,
response_model_exclude_unset=True,
response_model_exclude_none=True,
Expand Down Expand Up @@ -187,9 +188,11 @@ def register_post_search(self):
self.router.add_api_route(
name="Search",
path="/search",
response_model=(ItemCollection if not fields_ext else None)
if self.settings.enable_response_models
else None,
response_model=(
(ItemCollection if not fields_ext else None)
if self.settings.enable_response_models
else None
),
response_class=GeoJSONResponse,
response_model_exclude_unset=True,
response_model_exclude_none=True,
Expand All @@ -209,9 +212,11 @@ def register_get_search(self):
self.router.add_api_route(
name="Search",
path="/search",
response_model=(ItemCollection if not fields_ext else None)
if self.settings.enable_response_models
else None,
response_model=(
(ItemCollection if not fields_ext else None)
if self.settings.enable_response_models
else None
),
response_class=GeoJSONResponse,
response_model_exclude_unset=True,
response_model_exclude_none=True,
Expand All @@ -230,9 +235,9 @@ def register_get_collections(self):
self.router.add_api_route(
name="Get Collections",
path="/collections",
response_model=Collections
if self.settings.enable_response_models
else None,
response_model=(
Collections if self.settings.enable_response_models else None
),
response_class=self.response_class,
response_model_exclude_unset=True,
response_model_exclude_none=True,
Expand Down Expand Up @@ -280,9 +285,9 @@ def register_get_item_collection(self):
self.router.add_api_route(
name="Get ItemCollection",
path="/collections/{collection_id}/items",
response_model=ItemCollection
if self.settings.enable_response_models
else None,
response_model=(
ItemCollection if self.settings.enable_response_models else None
),
response_class=GeoJSONResponse,
response_model_exclude_unset=True,
response_model_exclude_none=True,
Expand Down Expand Up @@ -318,7 +323,7 @@ def register_core(self):
self.register_get_collection()
self.register_get_item_collection()

def customize_openapi(self) -> Optional[Dict[str, Any]]:
def customize_openapi(self) -> Optional[dict[str, Any]]:
"""Customize openapi schema."""
if self.app.openapi_schema:
return self.app.openapi_schema
Expand Down Expand Up @@ -346,7 +351,7 @@ async def ping():
self.app.include_router(mgmt_router, tags=["Liveliness/Readiness"])

def add_route_dependencies(
self, scopes: List[Scope], dependencies=List[Depends]
self, scopes: list[Scope], dependencies=list[Depends]
) -> None:
"""Add custom dependencies to routes.

Expand Down
1 change: 1 addition & 0 deletions stac_fastapi/api/stac_fastapi/api/config.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""Application settings."""

import enum


Expand Down
5 changes: 3 additions & 2 deletions stac_fastapi/api/stac_fastapi/api/errors.py
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
"""Error handling."""

import logging
from typing import Callable, Dict, Type, TypedDict
from typing import Callable, Type

from fastapi import FastAPI
from fastapi.exceptions import RequestValidationError
from starlette import status
from starlette.requests import Request
from starlette.responses import JSONResponse
from typing_extensions import TypedDict

from stac_fastapi.types.errors import (
ConflictError,
Expand Down Expand Up @@ -67,7 +68,7 @@ def handler(request: Request, exc: Exception):


def add_exception_handlers(
app: FastAPI, status_codes: Dict[Type[Exception], int]
app: FastAPI, status_codes: dict[Type[Exception], int]
) -> None:
"""Add exception handlers to the FastAPI application.

Expand Down
5 changes: 3 additions & 2 deletions stac_fastapi/api/stac_fastapi/api/middleware.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,9 @@
"""Api middleware."""

import re
import typing
from http.client import HTTP_PORT, HTTPS_PORT
from typing import List, Tuple
from typing import Tuple

from starlette.middleware.cors import CORSMiddleware as _CORSMiddleware
from starlette.types import ASGIApp, Receive, Scope, Send
Expand Down Expand Up @@ -126,7 +127,7 @@ def _get_header_value_by_name(
@staticmethod
def _replace_header_value_by_name(
scope: Scope, header_name: str, new_value: str
) -> List[Tuple[str]]:
) -> list[Tuple[str]]:
return [
(name, value)
for name, value in scope["headers"]
Expand Down
30 changes: 3 additions & 27 deletions stac_fastapi/api/stac_fastapi/api/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@
from typing import Optional, Type, Union

import attr
from fastapi import Body, Path
from fastapi import Path
from pydantic import BaseModel, create_model
from pydantic.fields import UndefinedType

from stac_fastapi.types.extension import ApiExtension
from stac_fastapi.types.search import (
Expand Down Expand Up @@ -44,31 +43,8 @@ def create_request_model(
# Handle POST requests
elif all([issubclass(m, BaseModel) for m in models]):
for model in models:
for k, v in model.__fields__.items():
field_info = v.field_info
body = Body(
None
if isinstance(field_info.default, UndefinedType)
else field_info.default,
default_factory=field_info.default_factory,
alias=field_info.alias,
alias_priority=field_info.alias_priority,
title=field_info.title,
description=field_info.description,
const=field_info.const,
gt=field_info.gt,
ge=field_info.ge,
lt=field_info.lt,
le=field_info.le,
multiple_of=field_info.multiple_of,
min_items=field_info.min_items,
max_items=field_info.max_items,
min_length=field_info.min_length,
max_length=field_info.max_length,
regex=field_info.regex,
extra=field_info.extra,
)
fields[k] = (v.outer_type_, body)
for k, field_info in model.model_fields.items():
fields[k] = (field_info.annotation, field_info)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@thomas-maschler sorry I should have checked before putting in the pull request it looks like most of our changes are the same. I found the field_info.annotation would leave out the Optional type unless I switched from typing List and Dict to the inbuilt list and dict.

return create_model(model_name, **fields, __base__=base_model)

raise TypeError("Mixed Request Model types. Check extension request types.")
Expand Down
7 changes: 4 additions & 3 deletions stac_fastapi/api/stac_fastapi/api/openapi.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
"""openapi."""

import warnings

from fastapi import FastAPI
Expand Down Expand Up @@ -43,9 +44,9 @@ async def patched_openapi_endpoint(req: Request) -> Response:
# Get the response from the old endpoint function
response: JSONResponse = await old_endpoint(req)
# Update the content type header in place
response.headers[
"content-type"
] = "application/vnd.oai.openapi+json;version=3.0"
response.headers["content-type"] = (
"application/vnd.oai.openapi+json;version=3.0"
)
# Return the updated response
return response

Expand Down
Loading
Loading