The Vision

[dbuscombe-usgs]

This is a toolbox for mapping coastal features in satellite imagery using deep-learning-based image segmentation.

Image segmentation involves classifying pixels (voxels) in imagery into one of two or more pre-defined categories. Deep learning is the application of neural networks with many layers. The layers are used to extract features from imagery that are used to estimate the class on a pixel by pixel basis.

CoastSeg combines CoastSat and Segmentation Zoo.

CoastSat functionality is used for

1. interfacing with Google Earth Engine to download and archive imagery using a geojson object
2. computing positional metadata
3. filtering clouds in imagery
4. convert geotiffs to jpegs
5. extract NIR, SWIR, and RGB bands and save them each to file

Segmentation Zoo functionality is used for

1. merging NIR, SWIR, and RGB bands and associated label images into ML-ready files
2. use those files to train a deep learning model or ensemble of models
3. use those models to carry out segmentation of sample imagery

This toolbox will attempt to be highly automated

Basic workflow

• User launches an interactive map server (localtileserver)
• User defines a Region-of-Interest (ROI), and other relevant spatial operations
• If ROI is large, ROI is subdivided into several smaller ROIs
• CoastSat functionality downloads imagery, archives, cloud masking, save tidal corrections
• Data converted to xarrays
• Zoo models act on xarray datacubes
• CoastSat models act on xarray datacubes
• Segmentations are mapped for visualization and further analysis
• CoastSat functionality creates transects, extracts time-series of shoreline at each transect
• Build in tools for
  • waterline mapping
  • other feature mapping, as dictated by projects and user contributions

Wishlist

• faster large-scale mapping
• mapping of any set of coastal features, landcovers, landusers, landforms, etc, that the user can make a Zoo model for
• everything carried out through a ipyleaflet based user interface and gui dialogs
• modified CoastSat Shoreline change analysis functions in a web map (ipyleaflet) environment
• no requirement on user-defined reference shoreline
• FES global tide model (like Inlet Tracker)

What do we need?

1. A local ipyleaflet tile server for data

   • Local Tile Server for Geospatial Rasters
   • all ipyleaflet and folium functionality for data exploration and retrieval

2. Cloud-based image retrieval

   • coastsat for image retrieval
   • also wxee for image retrieval and xarray support

3. Parallelism and modern geospatial formats

   • xarrays facilitate segmentation workflows in Dask
   • COGs

4. Doodler for geospatial imagery

   • interface imagery for doodler
   • wrapper utilities to convert labels to geospatial formats

5. Zoo for geospatial imagery

   • interface for preparing datasets for Zoo model training and evaluation
   • interface for applying trained models to geospatial imagery

6. Zoo models for MNDWI imagery

7. Lots of options for application of models (class logic)

[dbuscombe-usgs]

Add use cases / applications