Skip to content

Commit

Permalink
Release version 1.5.0, Merge pull request #751 from sentinel-hub/develop
Browse files Browse the repository at this point in the history
Release version 1.5.0
  • Loading branch information
zigaLuksic authored Sep 6, 2023
2 parents a65899e + d9dcffe commit 83f14c7
Show file tree
Hide file tree
Showing 234 changed files with 4,095 additions and 21,567 deletions.
32 changes: 11 additions & 21 deletions .github/workflows/ci_action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,26 +51,14 @@ jobs:
# caching the entire environment is faster when cache exists but slower for cache creation

- name: Install packages
run: |
pip install -r requirements-dev.txt --upgrade --upgrade-strategy eager
python install_all.py
pip install -r ml_tools/requirements-tdigest.txt
run: pip install .[DEV] --upgrade --upgrade-strategy eager

- name: Run pylint
run: make pylint
run: pylint eolearn

- name: Run mypy
if: success() || failure()
run: |
mypy \
core/eolearn/core \
coregistration/eolearn/coregistration \
geometry/eolearn/geometry \
features/eolearn/features \
io/eolearn/io \
mask/eolearn/mask \
ml_tools/eolearn/ml_tools \
visualization/eolearn/visualization
run: mypy eolearn

test-on-github:
runs-on: ubuntu-latest
Expand All @@ -95,24 +83,26 @@ jobs:
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
# cache: pip # uncomment when all requirements are in `pyproject.toml`
cache: pip

- name: Install packages
run: |
pip install -r requirements-dev.txt --upgrade --upgrade-strategy eager
python install_all.py -e
run: pip install -e .[DEV] --upgrade --upgrade-strategy eager

- name: Run full tests and code coverage
if: ${{ matrix.full_test_suite }}
run: |
sentinelhub.config \
--sh_client_id "${{ secrets.SH_CLIENT_ID }}" \
--sh_client_secret "${{ secrets.SH_CLIENT_SECRET }}"
pytest --cov --cov-report=term --cov-report=xml
if [ ${{ github.event_name }} == 'push' ]; then
pytest -m "not geopedia" --cov=eolearn --cov-report=term --cov-report=xml
else
pytest -m "not geopedia"
fi
- name: Run reduced tests
if: ${{ !matrix.full_test_suite }}
run: pytest -m "not sh_integration"
run: pytest -m "not sh_integration and not geopedia"

- name: Upload code coverage
if: ${{ matrix.full_test_suite && github.event_name == 'push' }}
Expand Down
3 changes: 1 addition & 2 deletions .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ run_sh_integration_tests:
- apt-get update
- apt-get install -y build-essential libgdal-dev graphviz proj-bin gcc libproj-dev libspatialindex-dev
script:
- pip install -r requirements-dev.txt --upgrade
- python install_all.py
- pip install .[DEV]
- sentinelhub.config --sh_client_id "$SH_CLIENT_ID" --sh_client_secret "$SH_CLIENT_SECRET" > /dev/null # Gitlab can't mask SH_CLIENT_SECRET in logs
- pytest -m sh_integration
4 changes: 2 additions & 2 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,13 +13,13 @@ repos:
- id: debug-statements

- repo: https://github.com/psf/black
rev: 23.3.0
rev: 23.7.0
hooks:
- id: black
language_version: python3

- repo: https://github.com/charliermarsh/ruff-pre-commit
rev: "v0.0.269"
rev: "v0.0.282"
hooks:
- id: ruff

Expand Down
34 changes: 31 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,32 @@
## [Version 1.4.2] - 2023-3-14
## [Version 1.5.0] - 2023-09-06

The release focuses on making `eo-learn` much simpler to install, reducing the number of dependencies, and improving validation of soundness of `EOPatch` data.

- `eo-learn` is now distributed as a single package. Installation of `eo-learn-mask` and similar is no longer necessary and users are warned when such installations are detected.
- Changes to `timestamps` and `bbox` attributes of `EOPatch` objects:
- `FeatureType.TIMESTAMPS` and `FeatureType.BBOX` have been deprecated, data should be accessed via attributes. Feature parsers no longer return these values (for instance when calling `EOPatch.get_features`).
- EOPatches without temporal information now have a timestamp value of `None`, whereas a timestamp value `[]` signifies that the EOPatch has a temporal dimension of 0.
- Introduced a `get_timestamps` method that will fail if `timestamps` are `None`. This can be used in cases where timestamps are assumed to be present (to avoid issues with type-checking and ill formed inputs).
- Loading, saving, and copying of EOPatches will take `timestamps` into account either when processing the full eopatch (i.e. `features=...`) or if the selection contains a temporal feature. The behavior can be controlled via the `load_timestamps`/`save_timestamps`/`copy_timestamps` parameter.
- Saving and loading of `FeatureType.META_INFO` now processes each feature as a separate file, allowing better filtering and preventing accidental overwriting.
- The default backend for `SpatialResizeTask` has been switched to `cv2` to reduce the number of dependencies.
- `eolearn.geometry.morphology` tasks now use `cv2` instead of `scikit-image` to reduce the number of dependencies. The task interfaces have been slightly adjusted.
- Improved reports:
- Exception grouping is now done by exception origin instead of exception message, resulting in shorter reports.
- Added execution time statistics per node
- `CloudMaskTask` has been restricted to mono-temporal predictions using the `s2cloudless` package. For the multi-temporal one check [here](https://github.com/sentinel-hub/eo-learn-examples/blob/main/extra-tasks/cloud_mask/cloud_mask.py).
- Certain tasks (for instance `SaveTask` and `LoadTask`) no longer pass arguments to the super-class via **kwargs in order to improve documentation and type-checking.
- `SaveTask` and `LoadTask` now raise `OSError` exceptions instead of `IOError`.
- Project-specific and outdated EOTasks were moved to extras or to the example repository [eo-learn-examples/extra-tasks](https://github.com/sentinel-hub/eo-learn-examples/tree/main/extra-tasks).
- The submodule `eolearn.features.bands_extraction` has been renamed to `eolearn.features.ndi`.
- The submodule `eolearn.ml_tools.extra.plotting` has been moved to `eolearn.visualization.utils`.
- Compression of EOPatch files has been hardcoded. The parameter `compression_level` has been deprecated and has no effect.
- Introduced experimental `zarr` support for loading/saving temporal slices of temporal features. The API might be changed in future releases.
- Limited `rasterio` to 1.3.7 due to an issue with importing rasters from AWS S3
- Updated examples, simplified tests, various improvements.


## [Version 1.4.2] - 2023-03-14

- Introduced support for Python 3.11.
- Removed support for Python 3.7.
Expand All @@ -13,7 +41,7 @@
- Various refactoring efforts and dependency improvements.
- Various improvements to tests and code.

## [Version 1.4.1] - 2023-3-14
## [Version 1.4.1] - 2023-03-14

- The codebase is now fully annotated and type annotations are mandatory for all new code.
- In the future `EOPatch` objects will **require** a valid `bbox`. For now the users are warned when no such value is provided.
Expand All @@ -27,7 +55,7 @@
- Various other minor improvements and deprecations.


## [Version 1.4.0] - 2023-1-20
## [Version 1.4.0] - 2023-01-20

- (**codebreaking**) Complete overhaul of `eolearn.coregistration`. See documentation for details.
- (**codebreaking**) Removed non-working HVPlot backend for `eolearn.visualization`.
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ For any question, feel free to contact us at [[email protected]](eoresear

We strive to provide high-quality working code, but bugs happen nevertheless.

When reporting a bug, please check [here][open-bug-list] whether the bug was already reported. If not, open an issue with the **bug** label and report the following information:
When reporting a bug, please check [the issue tracker][open-bug-list] whether the bug was already reported. If not, open an issue with the **bug** label and report the following information:

* Issue description
* How to reproduce the issue
Expand All @@ -36,7 +36,7 @@ This information helps us to reproduce, pinpoint, and fix the reported issue. If

## Feature Requests

Existing feature requests can be found [here][existing-feature-requests].
Existing feature requests can be found [in the issues][existing-feature-requests].

A new feature request can be created by opening a new issue with the **enhancement** label, and describing how the feature would benefit the **eo-learn** community. Providing an example use-case would help assessing the scope of the feature request.

Expand Down
5 changes: 0 additions & 5 deletions MANIFEST.in

This file was deleted.

186 changes: 10 additions & 176 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,187 +1,21 @@
# Makefile for creating a new release of the package and uploading it to PyPI

PYTHON = python3
PACKAGES = core coregistration features geometry io mask ml_tools visualization
PYLINT = pylint

.PHONY: $(PACKAGES:test)
.SILENT: pylint pylint-fast

help:
@echo "Use 'make upload-<package>' to upload the package to PyPi"
@echo "Use 'make pylint' to run pylint on the code of all subpackages"
@echo "Use 'make test-upload' to upload the package to testPyPi"
@echo "Use 'make upload' to upload the package to PyPi"

mypy:
mypy \
core/eolearn/core \
coregistration/eolearn/coregistration \
geometry/eolearn/geometry \
features/eolearn/features \
io/eolearn/io \
mask/eolearn/mask \
ml_tools/eolearn/ml_tools \
visualization/eolearn/visualization

pylint:
# Runs pylint on all subpackages consecutively and makes sure any error status code gets propagated.
export PYLINT_STATUS=0
for package in $(PACKAGES) ; do \
$(PYLINT) $$package/eolearn/$$package || export PYLINT_STATUS=$$?; \
done;
exit $$PYLINT_STATUS


pylint-fast:
# Runs pylint on all subpackages in parallel. Because of that output verdicts are not in the order of package
# names and this process cannot be interrupted.
for package in $(PACKAGES) ; do \
$(PYLINT) $$package/eolearn/$$package & \
done;
wait

.ONESHELL:
build-core:
cd core
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-coregistration:
cd coregistration
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-features:
cd features
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-geometry:
cd geometry
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-io:
cd io
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-mask:
cd mask
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-ml-tools:
cd ml_tools
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-visualization:
cd visualization
cp ../LICENSE LICENSE
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel
rm LICENSE

.ONESHELL:
build-abstract-package:
rm -r dist build | true
$(PYTHON) setup.py sdist bdist_wheel

upload-core: build-core
twine upload core/dist/*

upload-coregistration: build-coregistration
twine upload coregistration/dist/*

upload-features: build-features
twine upload features/dist/*

upload-geometry: build-geometry
twine upload geometry/dist/*

upload-io: build-io
twine upload io/dist/*

upload-mask: build-mask
twine upload mask/dist/*

upload-ml-tools: build-ml-tools
twine upload ml_tools/dist/*

upload-visualization: build-visualization
twine upload visualization/dist/*

upload-abstract-package: build-abstract-package
twine upload dist/*

upload-all: \
upload-core \
upload-coregistration \
upload-features \
upload-geometry \
upload-io \
upload-mask \
upload-ml-tools \
upload-visualization \
upload-abstract-package
upload:
rm -r dist | true
python -m build --sdist --wheel
twine upload --skip-existing dist/*

# For testing:

test-upload-core: build-core
twine upload --repository testpypi core/dist/*

test-upload-coregistration: build-coregistration
twine upload --repository testpypi coregistration/dist/*

test-upload-features: build-features
twine upload --repository testpypi features/dist/*

test-upload-geometry: build-geometry
twine upload --repository testpypi geometry/dist/*

test-upload-io: build-io
twine upload --repository testpypi io/dist/*

test-upload-mask: build-mask
twine upload --repository testpypi mask/dist/*

test-upload-ml-tools: build-ml-tools
twine upload --repository testpypi ml_tools/dist/*

test-upload-visualization: build-visualization
twine upload --repository testpypi visualization/dist/*

test-upload-abstract-package: build-abstract-package
twine upload --repository testpypi dist/*

test-upload-all: \
test-upload-core \
test-upload-coregistration \
test-upload-features \
test-upload-geometry \
test-upload-io \
test-upload-mask \
test-upload-ml-tools \
test-upload-visualization \
test-upload-abstract-package
test-upload:
rm -r dist | true
python -m build --sdist --wheel
twine upload --repository testpypi --skip-existing dist/*
Loading

0 comments on commit 83f14c7

Please sign in to comment.