From 792ad9a3df2a4822f03b3eda24b8f16d59622b32 Mon Sep 17 00:00:00 2001 From: Sharon Fitzpatrick Date: Wed, 27 Mar 2024 15:46:23 -0700 Subject: [PATCH] #239 implement reviewer 1 feedback, rewrote tide section, listed specific enhancements and clarified language --- paper/paper.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/paper/paper.md b/paper/paper.md index 5db245c7..7fc7e27a 100644 --- a/paper/paper.md +++ b/paper/paper.md @@ -42,17 +42,17 @@ This draft manuscript is distributed solely for purposes of courtesy review and # Summary -`CoastSeg` is an interactive browser-based program that aims to broaden the adoption of satellite-derived shoreline (SDS) detection and coastal landcover mapping workflows among coastal scientists and coastal resource management practitioners. SDS is a sub-field of coastal sciences that aims to detect and post-process a time-series of shoreline locations from publicly available satellite imagery [@turner2021satellite; @vitousek2023future]. `CoastSeg` is a python package installed via pip into a `conda` environment that serves as an toolkit for building custom SDS workflows. `CoastSeg` also provides full SDS workflow implementations via Jupyter notebooks and python scripts that call functions and classes in the core `CoastSeg` toolkit for specific workflows. Two fully functioning SDS workflows are already provided, and more could be added by collaborators in the SDS software community. All the codes, notebooks, scripts, and documentation are hosted on the `CoastSeg` GitHub repository. +`CoastSeg` is an interactive browser-based program that aims to broaden the adoption of satellite-derived shoreline (SDS) detection and coastal landcover mapping workflows among coastal scientists and coastal resource management practitioners. SDS is a sub-field of coastal sciences that aims to detect and post-process a time-series of shoreline locations from publicly available satellite imagery [@turner2021satellite; @vitousek2023future]. `CoastSeg` is a python package installed via pip into a `conda` environment that serves as an toolkit for building custom SDS workflows. `CoastSeg` also provides full SDS workflow implementations via Jupyter notebooks and python scripts that call functions and classes in the core `CoastSeg` toolkit for specific workflows. CoastSeg provides two fully functioning SDS workflows and its design allows for collaborators in the SDS software community to contribute additional workflows. All the codes, notebooks, scripts, and documentation are hosted on the `CoastSeg` GitHub repository. -So-called `instantaneous' SDS workflows, where shorelines are extracted from each individual satellite image rather than temporal composites [@bishop2021mapping], follow a basic recipe, namely 1) waterline estimation, where the 2D (x,y) location of the land-sea interface is determined, and 2) water-level correction, where the waterline location is mapped onto a shore-perpendicular transect, converted to a linear distance along that transect, then corrected for water level, and referenced to a particular elevation contour on the beach [@vos2019coastsat]. The resulting measurement is called a 'shoreline', and is an elevation-based measurement. Water level corrections typically only account for tide [@vos2019coastsat] but recently SDS workflows have incorporated both wave setup and runup correction, which are a function of the instantaneous wave field at the time of image acquisition [@konstantinou2023satellite; @vitousek2023future; @vitousek2023model,@castelle2021satellite]. +So-called `instantaneous' SDS workflows, where shorelines are extracted from each individual satellite image rather than temporal composites [@bishop2021mapping], follow a basic recipe, namely 1) waterline estimation, where the 2D (x,y) location of the land-sea interface is determined, and 2) water-level correction, where the waterline location is mapped onto a shore-perpendicular transect, converted to a linear distance along that transect, then corrected for water level, and referenced to a particular elevation contour on the beach [@vos2019coastsat]. The resulting measurement is called a 'shoreline' and it is the location that the waterline intersects a particular elevation datum. Water level corrections typically only account for tide [@vos2019coastsat] but recently SDS workflows have incorporated both wave setup and runup correction, which are a function of the instantaneous wave field at the time of image acquisition [@konstantinou2023satellite; @vitousek2023future; @vitousek2023model,@castelle2021satellite]. -`CoastSeg` has three broad aims. The first is to be an toolkit consisting functions that operate the core SDS workflow functionalities such as file input/output, image downloading, geospatial conversion, tidal model API handling, mapping 2D shorelines to 1D transect-based measurements, and numerous other workflows common to a basic SDS workflow, regardless of a particular waterline estimation workflow. This waterline detection algorithm will be crucial to the success of any SDS workflow because it is the step that identifies the the boundary between sea and land which serves as the basis for shoreline mapping. The idea behind the design of `CoastSeg` is that users could extend or customize functionality using scripts and notebooks. +`CoastSeg` has three broad aims. The first aim is to be an toolkit consisting functions that operate the core SDS workflow functionalities. This includes file input/output, image downloading, geospatial conversion, tidal model API handling, mapping 2D shorelines to 1D transect-based measurements, and numerous other functions common to a basic SDS workflow, regardless of a particular waterline estimation methodology. This waterline detection algorithm will be crucial to the success of any SDS workflow because it is the step that identifies the the boundary between sea and land which serves as the basis for shoreline mapping. The idea behind the design of `CoastSeg` is that users could extend or customize functionality using scripts and notebooks. -The second aim of `CoastSeg` is therefore to provide fully functioning SDS implementations in an accessible browser notebook format. Our principal objective to date has been to re-implement and improve upon a popular existing toolbox, `CoastSat` [@vos2019coastsat], allowing the user to carry out the well-established `CoastSat` SDS workflow with a well-supported literature [@castelle2021satellite; @castelle2022primary; @vos2023pacific; @vos2023benchmarking; @warrick2023large; @konstantinou2023satellite; @vitousek2023model; @mclean202350; @vandenhove2024secular], but in a more accessible and convenient way within the `CoastSeg` platform. In order to achieve this, we developed `CoastSat-package` [@voscoastsat], a python package that is installed into the `CoastSeg` `conda` environment. `CoastSat-package` contains re-implemented versions of the original CoastSat codes, addresses the lack of pip or conda installability of CoastSat, and isolates the CoastSeg-specific enhancements from the original `CoastSat` code. The `CoastSeg` re-implementation of the `CoastSat` workflow is end-to-end within a single notebook. That notebook allows the user to, among other tasks: a) define a region of interest on a webmap and upload geospatial vector format files; b) define, download and post-process satellite imagery; c) identify waterlines in that imagery using the `CoastSat` method [@vos2019coastsat]; d) correct those waterlines to elevation-based shorelines using tidal elevation-datum corrections provided through interaction with the pyTMD [@alley2017pytmd] API; and e) download output files in a variety of modern geospatial and other formats for subsequent analysis. +The second aim of `CoastSeg` is therefore to provide fully functioning SDS implementations in an accessible browser notebook format. Our principal objective to date has been to re-implement and improve upon a popular existing toolbox, `CoastSat` [@vos2019coastsat], allowing the user to carry out the well-established `CoastSat` SDS workflow with a well-supported literature [@castelle2021satellite; @castelle2022primary; @vos2023pacific; @vos2023benchmarking; @warrick2023large; @konstantinou2023satellite; @vitousek2023model; @mclean202350; @vandenhove2024secular], but in a more accessible and convenient way within the `CoastSeg` platform. In order to achieve this, we developed `CoastSat-package` [@voscoastsat], a python package that is installed into the `CoastSeg` `conda` environment. `CoastSat-package` contains re-implemented versions of the original CoastSat codes, addresses the lack of pip or conda installability of CoastSat, and isolates the CoastSeg-specific enhancements from the original `CoastSat` code. The `CoastSeg` re-implementation of the `CoastSat` workflow is end-to-end within a single notebook. That notebook allows the user to, among other tasks: a) define a region of interest on a webmap and upload geospatial vector format files; b) define, download and post-process satellite imagery; c) identify waterlines in that imagery using the `CoastSat` method [@vos2019coastsat]; d) correct those waterlines to elevation-based shorelines using tidal elevation-datum corrections provided through interaction with the pyTMD [@alley2017pytmd] API; and e) save output files in a variety of modern geospatial and other formats for subsequent analysis. Additionally, CoastSeg's toolkit-based design enables it to run as non-interactive scripts, catering to larger scale shoreline analysis projects.This flexibility ensures that CoastSeg can accommodate a wide range of research needs, from detailed, interactive exploration to extensive, automated analyses. -The third and final aim of `CoastSeg` is to implement a method to carry out SDS workflows in experimental and collaborative contexts, which aids both oversight and reproducibility as well as practical needs based on division of labor. We do this using `sessions`, a mechanism for saving the current state of the application into a session folder. This folder contains all necessary inputs, outputs, and references to downloaded data used to generate the results. `Sessions` allow users to iteratively experiment with different combinations of settings and makes `CoastSeg` fully reproducible because everything needed to reproduce the session is saved to the folder. Users can share their `sessions` with others, enabling peers to replicate experiments, build upon previous work, or access data downloaded by someone else. This simplifies handovers to new users from existing users, simplifies teaching of the program, and encourages collective experimentation which may result in better shoreline data. +The third and final aim of `CoastSeg` is to implement a method to carry out SDS workflows in experimental and collaborative contexts, which aids both oversight and reproducibility as well as practical needs based on division of labor. We do this using `sessions`, a mechanism for saving the current state of the application into a session's folder. This folder contains all necessary inputs, outputs, and references to downloaded data used to generate the results. `Sessions` allow users to iteratively experiment with different combinations of settings and makes `CoastSeg` fully reproducible because everything needed to reproduce the session is saved to the folder. Users can share their `sessions` with others, enabling peers to replicate experiments, build upon previous work, or access data downloaded by someone else. This simplifies handovers to new users from existing users, simplifies teaching of the program, and encourages collective experimentation which may result in better shoreline data. -`CoastSeg` is also designed to be extendable, serving as a hub that hosts alternative SDS workflows and similar workflows that can be encoded in a Jupyter notebook built upon the `CoastSeg` and `CoastSat-package` core functionalities. Additional notebooks can be designed to carry out shoreline extraction and coastal landcover mapping using alternative methods. We provide an example of an alternative SDS workflow based on a deep-learning based semantic segmentation model, that is briefly summarized at the end of this paper. To implement a custom waterline detection workflow the originator of that workflow would contribute a new Jupyter notebook, and also likely contribute to the `CoastSeg` source code to add their specific waterline detection algorithm, which could be called in their notebook. +`CoastSeg` is also designed to be extendable, serving as a hub that hosts alternative SDS workflows and similar workflows that can be encoded in a Jupyter notebook built upon the `CoastSeg` and `CoastSat-package` core functionalities. Additional notebooks can be designed to carry out shoreline extraction and coastal landcover mapping using alternative methods. We provide an example of an alternative SDS workflow based on a deep-learning based semantic segmentation model that is briefly summarized at the end of this paper. To implement a custom waterline detection workflow the originator of that workflow would contribute new Jupyter notebook, and add their specific waterline detection algorithm to the `CoastSeg` source code, so it could be used in their notebook's implementation. @@ -60,7 +60,7 @@ The third and final aim of `CoastSeg` is to implement a method to carry out SDS Coastal scientists and resource managers now have access to extensive collections of satellite data spanning more than four decades. However, it's only in recent years that advancements in algorithms, machine learning, and deep learning have enabled the automation of processing this satellite imagery to accurately identify and map shorelines from imagery, a process known as Satellite-Derived Shorelines, or SDS. SDS workflows [@garcia2015evaluating; @almonacid2016evaluation] are gaining rapidly in popularity, and in particular since the publication of the open-source implementation of the `CoastSat` workflow [@coastsat] for instantaneous SDS in 2018 [@vos2019coastsat]. Existing open-source software for SDS often require the user to navigate between platforms (non-reproducible elements), develop custom code, and/or engage in substantial manual effort. -We sought to build a platform that not only allowed the user to adopt the `CoastSat` workflow in a re-implementation than within a single Jupyter notebook, in a quicker, and in a more seamless manner, but also one that facilitates experimentation with the many settings that can govern shoreline accuracy, extent, and number. Further, `CoastSeg` has been designed specifically to host alternative SDS workflows, recognizing that it is a nascent field of coastal science, and the optimal methodologies for all coastal environments and sources of imagery are yet to be established. Therefore `CoastSeg` will provide a means with which to extract shorelines using multiple methods and adopt the one that most suits their needs, or implement a new methods. +We built CoastSeg with the aim of enhancing the CoastSat workflow. Our design streamlines the entire shoreline extraction process, thus facilitating a more efficient experimental approach to determine the optimal combination of settings to extract the greatest number of accurate shorelines. CoastSeg achieves these improvements through several key advancements: it ensures reproducible sessions for consistent comparison and analysis; introduces additional filtering mechanisms to refine results; and provides an interactive user webmap that allows the users to users to view the quality of the extracted shorelines. Further, `CoastSeg` has been designed specifically to host alternative SDS workflows, recognizing that it is a nascent field of coastal science, and the optimal methodologies for all coastal environments and sources of imagery are yet to be established. Therefore `CoastSeg` provides a means with which to extract shorelines using multiple methods and adopt the one that most suits their needs, or implement a new methods. We summarize the needs met by the `CoastSeg` project as follows: @@ -86,7 +86,7 @@ We summarize the needs met by the `CoastSeg` project as follows: At a high level, `CoastSeg` is designed to be an accessible and extendable hub for both `CoastSat`-based and alternate workflows, each of which is implemented in a single notebook. The user is therefore presented with a single menu of notebooks, each of which calls on a common set of core functionalities provided by `CoastSeg` and `coastsat-package`, and exporting data to common file formats and conventions. -`CoastSeg` is installable as a pip package into a `conda` environment. `CoastSeg` notebooks are accessed from GitHub. We also created a pip package for the `Coastsat` workflow in order to a) improve the `CoastSat` method's software implementation without affecting the parent repository, and b) to install as a pip package into a `conda` environment, rather than duplicate code from CoastSat. +`CoastSeg` is installable as a package into a `conda` environment. `CoastSeg` notebooks are accessed from GitHub. We also created a pip package for the `CoastSat` workflow we named `CoastSat-package` in order to a) improve the `CoastSat` method's software implementation without affecting the parent repository, and b) to install as a package into a `conda` environment, rather than duplicate code from CoastSat. `CoastSeg` is built with a object-oriented architecture, where elements required by the `CoastSat` workflow such as Regions of Interest, reference shorelines, and transects are represented as distinct objects on the map. Each class stores data specific to that feature type as well as encompassing methods for styling the feature on the map, downloading default features, and executing various post-processing functions. @@ -98,7 +98,7 @@ SDS workflows require manipulating various settings in order to extract optimal ### Accessibility -`CoastSeg` facilitates entirely browser-based workflows with an interactive webmap and `ipywidget` controls. It interfaces with the Zenodo API to download reference shorelines for any location in the world, organized into 5x5 degree chunks in GeoJSON format [@buscombe_2023_7786276] as well as transects, themselves providing beachface slope metadata [@buscombe_2023_8187949] available when users hover over each transect with their cursor. We have implemented rigorous error handling using developer log files, user report files, and informative error messages that suggest problem fixes. We have also provided a set of utility scripts for common data input/output tasks, often the result of specific requests from our software testers (see Acknowledgments). In addition to a project wiki and improved documentation, we have researched minimum, maximum, and recommended values for all settings, and have provided visual project management aids. +`CoastSeg` facilitates entirely browser-based workflows with an interactive webmap and `ipywidget` controls. It interfaces with the Zenodo API to download reference shorelines for any location in the world, organized into 5x5 degree chunks in GeoJSON format [@buscombe_2023_7786276] as well as transects, themselves providing beachface slope metadata [@buscombe_2023_8187949] available when users hover over each transect with their cursor. We have improved the reliability of `CoastSeg` through rigorous error handling, which includes developer log files for in-depth diagnostics, user report files for transparency, and detailed error messages that provide guidance for troubleshooting and problem resolution. We have also provided a set of utility scripts for common data input/output tasks, often the result of specific requests from our software testers (see Acknowledgments). In addition to a project wiki and improved documentation, we have researched minimum, maximum, and recommended values for all settings, set suggested default values, and have provided visual project management aids. ### Performance @@ -110,7 +110,9 @@ We added helpful workflow components such as image filtering options; for exampl ### Tide -Tidal correction (Figure 1) of shorelines involves estimating the tide height for any location and time using the `pyTMD` API [@alley2017pytmd] to model the tide. `pyTMD` provides an accessible script for the widely used FES14 [@lyard2021fes2014] tidal model data access, and includes several models other than FES14 including polar-specific models. We created an automated workflow that splits the FES2014 model data into 11 global regions (an idea adopted from [@krause2021dea]). This allows the program to access only a subset of the data, facilitating fast tide estimates (in minutes rather than hours for multi-decadal satellite time-series). +The CoastSat methodology for applying tide correction to shoreline positions involved a multi-step process. First the user would need to independently download and configure the FES14 [@lyard2021fes2014] tide model,a widely recognized tidal model. After configuring the tide model, users would then generate tide estimates at 15-minute intervals for a single location within their Region of Interest (ROI) across the entire satellite imagery time series. The tide estimate closest to the time of shoreline detection was used to adjust the shoreline position. This method, while comprehensive, was time-consuming, potentially requiring hours to generate all necessary tide estimates. + +In contrast, CoastSeg introduces a significant improvement to this process by leveraging the pyTMD API [@alley2017pytmd] for a more streamlined and accurate approach to tidal correction (Figure 1). pyTMD facilitates downloading a variety of tide models, including FES14 and models specific to polar regions, and automates tide estimations. We provide an automated work that downloads and subdivides the FES2014 model data into 11 global regions (an idea adopted from [@krause2021dea]). This subdivision allows the program to access only relevant subsets of data, drastically reducing the time required to estimate tides—from hours to minutes for multi-decadal satellite time series. Furthermore, CoastSeg calculates tide estimates for each transect corresponding to the times shorelines were detected. This ensures tide corrections are based on precise temporal and spatial matches, enhancing the accuracy of shoreline position adjustments. ![Schematic of the SDS workflows currently available in ``CoastSeg``. a) ``CoastSat`` workflow; b) ``Zoo`` workflow.](figs/coastseg_figure_2.png){#sylt width="100%"}