Split into COG and NetCDF sub-packages

.github/workflows/continuous-integration.yml

@@ -28,6 +28,7 @@ jobs:
   python-matrix:
     name: python-matrix
     runs-on: ubuntu-latest
+    timeout-minutes: 30
     strategy:
       matrix:
         python-version: ["3.8", "3.9", "3.10"]

CHANGELOG.md

@@ -11,15 +11,16 @@ number as needed.
 
 ### Added
 
-- Nothing.
+- Dedicated NetCDF and COG Collections
+- COG Tiler
 
 ### Deprecated
 
 - Nothing.
 
 ### Removed
 
-- Nothing.
+- Single Item creation containing worldwide COGs and NetCDF assets.
 
 ### Fixed
 

README.md

@@ -11,77 +11,65 @@
 - STAC extensions used:
   - [classification](https://github.com/stac-extensions/classification/)
   - [datacube](https://github.com/stac-extensions/datacube/)
+  - [grid](https://github.com/stac-extensions/grid/)
+  - [item-assets](https://github.com/stac-extensions/item-assets)
   - [processing](https://github.com/stac-extensions/processing/)
-  - [proj](https://github.com/stac-extensions/projection/)
+  - [projection](https://github.com/stac-extensions/projection/)
   - [raster](https://github.com/stac-extensions/raster/)
-  - [version](https://github.com/stac-extensions/version/)
+  - [scientific](https://github.com/stac-extensions/scientific)
 - Extra fields:
-  - None
+  - `esa_cci_lc:version`: Land cover product version.
+- [Browse the example in human-readable form](https://radiantearth.github.io/stac-browser/#/external/raw.githubusercontent.com/stactools-packages/esa-cci-lc/main/examples/catalog.json)
 
-A stactools package for ESA's Climate Change Initiative (CCI) Land Cover (LC)
-product.
+## Background
 
-This dataset provides global maps describing the land surface classes,
+A stactools package for ESA's Climate Change Initiative (CCI) Land Cover (LC)
+product. The ESA CCI LC dataset provides global maps describing the land surface classes,
 which have been defined using the United Nations Food and Agriculture
 Organization's (UN FAO) Land Cover Classification System (LCCS).
 In addition to the land cover (LC) maps, four quality flags are produced to
 document the reliability of the classification and change detection.
-In order to ensure continuity, these land cover maps are consistent with the
-series of global annual LC maps from the 1990s to 2015 produced by the
-European Space Agency (ESA) Climate Change Initiative (CCI).
 
-This package can generate STAC files from netCDF files and that either link to
-the original netCDF files or to Cloud-Optimized GeoTiff (COG) files.
+Two STAC Collections, and corresponding Items, can be generated with this package:
 
-## STAC Examples
+1. esa-cci-lc: A Collection of COG tiles generated from the source NetCDF data files.
+2. esa-cci-lc-netcdf: A Collection describing the source NetCDF data files.
 
-- [Collection](examples/collection.json)
-- [Item](examples/item.json)
-- [Browse the example in human-readable form](https://radiantearth.github.io/stac-browser/#/external/raw.githubusercontent.com/stactools-packages/esa-cci-lc/main/examples/collection.json)
+If you are interested in creating single STAC Items for each source NetCDF file along with worldwide COGs (not tiled), see release [v0.1.0](https://github.com/stactools-packages/esa-cci-lc/releases/tag/v0.1.0).
 
-## Installation
-
-```shell
-pip install stactools-esa-cci-lc
-```
-
-## Command-line Usage
-
-Use `stac esa-cci-lc --help` to see all subcommands and options.
+## STAC Examples
 
-### Collection
+- Collections
+  - [Tiled COGs](examples/esa-cci-lc/collection.json)
+  - [Source NetCDF](examples/esa-cci-lc-netcdf/collection.json)
 
-Create a collection:
+- Items
+  - [Tiled COGs](examples/esa-cci-lc/C3S-LC-L4-LCCS-Map-300m-P1Y-2018-v2.1.1-N79W180/C3S-LC-L4-LCCS-Map-300m-P1Y-2018-v2.1.1-N79W180.json)
+  - [Source NetCDF](examples/esa-cci-lc-netcdf/C3S-LC-L4-LCCS-Map-300m-P1Y-2018-v2.1.1/C3S-LC-L4-LCCS-Map-300m-P1Y-2018-v2.1.1.json)
 
-```shell
-stac esa-cci-lc create-collection collection.json
-```
+The example Collections and Items in the `examples` directory can be created by running `./scripts/create_examples.py`.
 
-Get information about all options for collection creation:
+## Installation
 
 ```shell
-stac esa-cci-lc create-collection --help
+pip install stactools-esa-cci-lc
 ```
 
-### Item
+## Command-line Usage
 
-Create an item with netCDF and COG assets:
+To create the NetCDF Collection:
 
 ```shell
-stac esa-cci-lc create-item /path/to/source/file.nc item.json --collection collection.json
+stac esa-cci-lc netcdf create-collection collection.json
 ```
 
-Create an item with only COG assets:
+To convert a NetCDF to tiled COGs and create an Item for each tile:
 
 ```shell
-stac esa-cci-lc create-item /path/to/source/file.nc item.json --collection collection.json --nonetcdf TRUE
+stac esa-cci-lc cog create-items /path/to/source/file.nc /path/to/output/directory
 ```
 
-Get information about all options for item creation:
-
-```shell
-stac esa-cci-lc create-item --help
-```
+Use `stac esa-cci-lc --help` to see all subcommands and options.
 
 ## Contributing
 
@@ -105,18 +93,3 @@ To run the tests:
 ```shell
 pytest -vv
 ```
-
-### Tests
-
-Due to the large file sizes, the data files for the tests are missing in this repository.
-To run the tests, please download the files for 1992, 2015, 2016 and 2020 from the
-[Climate Data Store](https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-land-cover?tab=form).
-You can download them all at once in a ZIP or gzipped TAR file and then extract the `.nc` files
-to the `tests/data-files/external` folder.
-
-The CI doesn't run any tests for item generation due to the missing data files.
-Please ensure you've tested the changes against the data files locally!
-
-Also, due to the long processing time of the tests with COGs conversion (in total 3-4 hours on 
-my local machine), the tests by default run without COG processing (i.e. `--nocog` is enabled).
-To run the tests with COG generation, please run `pytest -vv --withcog`.

environment.yml

@@ -6,4 +6,3 @@ dependencies:
   - conda-forge::gdal>=3.4
   - conda-forge::geos>=3.3
   - conda-forge::rasterio>=1.3
-  - conda-forge::libstdcxx-ng # gdal dependency. Make sure it's from the same channel as gdal