diff --git a/.Dockerignore b/.Dockerignore new file mode 100644 index 0000000..44bd80c --- /dev/null +++ b/.Dockerignore @@ -0,0 +1,24 @@ +# Mypy +.mypy_cache +.pytest_cache + +# MacOS +.DS_Store +.DS_Store? + +# vscode +.vscode + +# Git +.git +.github + +# Environment +.env + +# Ignore build artifacts +build/ +*.egg-info/ + +# Ignore Jupyter checkpoints +.ipynb_checkpoints/ diff --git a/CHANGELOG.md b/CHANGELOG.md index 98c40ad..5153553 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/) and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html). +## [0.0.3] + +### Added + +- Python 3.13 support +- Updated dockerimage to ensure on login the conda environment is activated +- Instructions in the README for OPERA delivery. +- A `.Dockerignore` file to remove extraneous files from the docker image +- Allow `/home/ops` directory in Docker image to be open to all users + ## [0.0.2] ### Added diff --git a/Dockerfile b/Dockerfile index 42c3bcf..c1b1de8 100644 --- a/Dockerfile +++ b/Dockerfile @@ -29,6 +29,9 @@ WORKDIR /home/ops COPY --chown=dist_user:dist_user environment.yml /home/ops/dist-s1/environment.yml COPY --chown=dist_user:dist_user . /home/ops/dist-s1 +# Ensure all files are read/write by the user +RUN chmod -R 777 /home/ops + # Create the environment with mamba RUN mamba env create -f /home/ops/dist-s1/environment.yml && \ conda clean -afy @@ -37,5 +40,7 @@ RUN mamba env create -f /home/ops/dist-s1/environment.yml && \ RUN echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.profile && \ echo "conda activate dist-s1-env" >> ~/.profile +RUN echo "conda activate dist-s1-env" >> ~/.bashrc + # Install repository with pip RUN python -m pip install --no-cache-dir /home/ops/dist-s1 diff --git a/README.md b/README.md index 90ee518..abe7edf 100644 --- a/README.md +++ b/README.md @@ -6,9 +6,29 @@ [![Conda version](https://img.shields.io/conda/vn/conda-forge/dist_s1)](https://anaconda.org/conda-forge/dist_s1) [![Conda platforms](https://img.shields.io/conda/pn/conda-forge/dist_s1)](https://anaconda.org/conda-forge/dist_s1) -This is the workflow that generates OPERA's DIST-S1 product. This workflow is designed to delineate *generic* disturbance from a time-series of OPERA Radiometric and Terrain Corrected Sentinel-1 (RTC-S1) products. +This is the workflow that generates OPERA's DIST-S1 product. This workflow is designed to delineate *generic* disturbance from a time-series of OPERA Radiometric and Terrain Corrected Sentinel-1 (OPERA RTC-S1) products. The output DIST-S1 product is resampled to a 30 meter Military Grid Reference System (MGRS) tile. -## Install +Currently, this workflow is just *scaffolding*. It is not ready for use! + +## Installation + +### Using `pip` + +We recommend using the mamba/conda package manager to install the DIST-S1 workflow, manage the environment, and install the dependencies. + +``` +mamba update -f environment.yml +pip install dist-s1 # update to conda when it is ready on conda-forge +conda activate dist-s1-env +python -m ipykernel install --user --name dist-s1-env +``` + +The last command is optional, but will allow this project to be imported into a Jupyter notebook. + + +### Development Installation + +As above, we recommend using the mamba/conda package manager to install the DIST-S1 workflow, manage the environment, and install the dependencies. ``` mamba update -f environment.yml @@ -19,12 +39,24 @@ python -m ipykernel install --user --name dist-s1-env ## Usage +There are two entrypoints for the DIST-S1 workflow: + +1. `dist-s1 run_sas` - This is the primary entrypoint for Science Data System (SDS) operations in which this library is viewed as the Science Application Software (SAS) for DIST-S1 within JPL's Hybrid Science Data System (HySDS). +2. `dist-s1 run` (not implemented yet) - This is the simplified entrypoint for scientific and research users (non-SDS), allowing for the localization of data from publicly available data sources with more human readable inputs. + +It is worth noting that the SDS workflow (`dist-s1 run_sas`) is *not* user friendly requiring the explicit specification of the numerous input RTC-S1 datasets (nominally there are 100s of these files required for the generation of a single DIST-S1 product over an MGRS tile). The `dist-s1 run` entrypoint has far fewer inputs and is designed to be human-operable. Specifically, the `dist-s1 run` takes care of the localization and accounting of the all the necessary input RTC-S1 datasets. + +### The `dist-s1 run_sas` Entrypoint + ``` -dist-s1 --runconfig_yml_path +dist-s1 run_sas --runconfig_yml_path ``` See `tests/test_main.py` for an example of how to use the CLI with sample data. +### The `dist-s1 run` Entrypoint + +This is not yet implemented. ## Docker @@ -35,18 +67,43 @@ docker pull ghcr.io/asf-hyp3/dist-s1: ``` Where `` is the semantic version of the release you want to download. +Notes: +- our image does not currently support the `arm64` (i.e. Mac M1) architecture. Therefore, you will need to build the image from the Dockerfile yourself. +- Currently, the image is still under development and we will likely update it to ensure compatibility with GPU processing. + ### Building the Docker Image Locally -Make sure you have Docker installed (e.g. for MacOS: https://docs.docker.com/desktop/setup/install/mac-install/) +Make sure you have Docker installed for [Mac](https://docs.docker.com/desktop/setup/install/mac-install/) or [Windows](https://docs.docker.com/desktop/setup/install/windows-install/). We call the docker image `dist_s1_img` for the remainder of this README. ``` -docker build -f Dockerfile -t dist_s1_img . +docker build -f Dockerfile -t dist-s1 . ``` ### Running the Container Interactively To run the container interactively: ``` -docker run -ti dist_s1_img +docker run -ti dist-s1 ``` Within the container, you can run the CLI commands and the test suite. + + +# Delivery Instructions + +There are certain releases associated with OPERA project deliveries. Here we provide instructions for how to run and verify the DIST-S1 workflow. + +We have included sample input data, associated a Docker image via the Github registry, and run tests via github actions all within this repository. + +``` +docker pull ghcr.io/opera-adt/dist-s1 +``` +If a specific version is required (or assumed for a delivery), then you use `docker pull ghcr.io/opera-adt/dist-s1:` e.g. +``` +docker pull ghcr.io/opera-adt/dist-s1:0.0.4 +``` +The command will pull the latest released version of the Docker image. To run the test suite, run: +``` +docker run ghcr.io/opera-adt/dist-s1 bash -l -c 'cd dist-s1 && pytest tests' +``` + + diff --git a/pyproject.toml b/pyproject.toml index 1e79a35..dfc056c 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -16,6 +16,7 @@ classifiers = [ "Intended Audience :: Science/Research", "Programming Language :: Python :: 3", "Programming Language :: Python :: 3.12", + "Programming Language :: Python :: 3.13", "Programming Language :: Python", "Topic :: Scientific/Engineering", ]