Skip to content

Commit

Permalink
Merge pull request #10 from opera-adt/dev
Browse files Browse the repository at this point in the history
v0.0.3
  • Loading branch information
cmarshak authored Nov 23, 2024
2 parents 12a9ca1 + 0fc5d18 commit f596dbd
Show file tree
Hide file tree
Showing 5 changed files with 103 additions and 6 deletions.
24 changes: 24 additions & 0 deletions .Dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Mypy
.mypy_cache
.pytest_cache

# MacOS
.DS_Store
.DS_Store?

# vscode
.vscode

# Git
.git
.github

# Environment
.env

# Ignore build artifacts
build/
*.egg-info/

# Ignore Jupyter checkpoints
.ipynb_checkpoints/
10 changes: 10 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,16 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [PEP 440](https://www.python.org/dev/peps/pep-0440/)
and uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [0.0.3]

### Added

- Python 3.13 support
- Updated dockerimage to ensure on login the conda environment is activated
- Instructions in the README for OPERA delivery.
- A `.Dockerignore` file to remove extraneous files from the docker image
- Allow `/home/ops` directory in Docker image to be open to all users

## [0.0.2]

### Added
Expand Down
5 changes: 5 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,9 @@ WORKDIR /home/ops
COPY --chown=dist_user:dist_user environment.yml /home/ops/dist-s1/environment.yml
COPY --chown=dist_user:dist_user . /home/ops/dist-s1

# Ensure all files are read/write by the user
RUN chmod -R 777 /home/ops

# Create the environment with mamba
RUN mamba env create -f /home/ops/dist-s1/environment.yml && \
conda clean -afy
Expand All @@ -37,5 +40,7 @@ RUN mamba env create -f /home/ops/dist-s1/environment.yml && \
RUN echo ". /opt/conda/etc/profile.d/conda.sh" >> ~/.profile && \
echo "conda activate dist-s1-env" >> ~/.profile

RUN echo "conda activate dist-s1-env" >> ~/.bashrc

# Install repository with pip
RUN python -m pip install --no-cache-dir /home/ops/dist-s1
69 changes: 63 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,29 @@
[![Conda version](https://img.shields.io/conda/vn/conda-forge/dist_s1)](https://anaconda.org/conda-forge/dist_s1)
[![Conda platforms](https://img.shields.io/conda/pn/conda-forge/dist_s1)](https://anaconda.org/conda-forge/dist_s1)

This is the workflow that generates OPERA's DIST-S1 product. This workflow is designed to delineate *generic* disturbance from a time-series of OPERA Radiometric and Terrain Corrected Sentinel-1 (RTC-S1) products.
This is the workflow that generates OPERA's DIST-S1 product. This workflow is designed to delineate *generic* disturbance from a time-series of OPERA Radiometric and Terrain Corrected Sentinel-1 (OPERA RTC-S1) products. The output DIST-S1 product is resampled to a 30 meter Military Grid Reference System (MGRS) tile.

## Install
Currently, this workflow is just *scaffolding*. It is not ready for use!

## Installation

### Using `pip`

We recommend using the mamba/conda package manager to install the DIST-S1 workflow, manage the environment, and install the dependencies.

```
mamba update -f environment.yml
pip install dist-s1 # update to conda when it is ready on conda-forge
conda activate dist-s1-env
python -m ipykernel install --user --name dist-s1-env
```

The last command is optional, but will allow this project to be imported into a Jupyter notebook.


### Development Installation

As above, we recommend using the mamba/conda package manager to install the DIST-S1 workflow, manage the environment, and install the dependencies.

```
mamba update -f environment.yml
Expand All @@ -19,12 +39,24 @@ python -m ipykernel install --user --name dist-s1-env

## Usage

There are two entrypoints for the DIST-S1 workflow:

1. `dist-s1 run_sas` - This is the primary entrypoint for Science Data System (SDS) operations in which this library is viewed as the Science Application Software (SAS) for DIST-S1 within JPL's Hybrid Science Data System (HySDS).
2. `dist-s1 run` (not implemented yet) - This is the simplified entrypoint for scientific and research users (non-SDS), allowing for the localization of data from publicly available data sources with more human readable inputs.

It is worth noting that the SDS workflow (`dist-s1 run_sas`) is *not* user friendly requiring the explicit specification of the numerous input RTC-S1 datasets (nominally there are 100s of these files required for the generation of a single DIST-S1 product over an MGRS tile). The `dist-s1 run` entrypoint has far fewer inputs and is designed to be human-operable. Specifically, the `dist-s1 run` takes care of the localization and accounting of the all the necessary input RTC-S1 datasets.

### The `dist-s1 run_sas` Entrypoint

```
dist-s1 --runconfig_yml_path <path/to/runconfig.yml>
dist-s1 run_sas --runconfig_yml_path <path/to/runconfig.yml>
```

See `tests/test_main.py` for an example of how to use the CLI with sample data.

### The `dist-s1 run` Entrypoint

This is not yet implemented.

## Docker

Expand All @@ -35,18 +67,43 @@ docker pull ghcr.io/asf-hyp3/dist-s1:<tag>
```
Where `<tag>` is the semantic version of the release you want to download.

Notes:
- our image does not currently support the `arm64` (i.e. Mac M1) architecture. Therefore, you will need to build the image from the Dockerfile yourself.
- Currently, the image is still under development and we will likely update it to ensure compatibility with GPU processing.

### Building the Docker Image Locally

Make sure you have Docker installed (e.g. for MacOS: https://docs.docker.com/desktop/setup/install/mac-install/)
Make sure you have Docker installed for [Mac](https://docs.docker.com/desktop/setup/install/mac-install/) or [Windows](https://docs.docker.com/desktop/setup/install/windows-install/). We call the docker image `dist_s1_img` for the remainder of this README.

```
docker build -f Dockerfile -t dist_s1_img .
docker build -f Dockerfile -t dist-s1 .
```

### Running the Container Interactively

To run the container interactively:
```
docker run -ti dist_s1_img
docker run -ti dist-s1
```
Within the container, you can run the CLI commands and the test suite.


# Delivery Instructions

There are certain releases associated with OPERA project deliveries. Here we provide instructions for how to run and verify the DIST-S1 workflow.

We have included sample input data, associated a Docker image via the Github registry, and run tests via github actions all within this repository.

```
docker pull ghcr.io/opera-adt/dist-s1
```
If a specific version is required (or assumed for a delivery), then you use `docker pull ghcr.io/opera-adt/dist-s1:<version>` e.g.
```
docker pull ghcr.io/opera-adt/dist-s1:0.0.4
```
The command will pull the latest released version of the Docker image. To run the test suite, run:
```
docker run ghcr.io/opera-adt/dist-s1 bash -l -c 'cd dist-s1 && pytest tests'
```


1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ classifiers = [
"Intended Audience :: Science/Research",
"Programming Language :: Python :: 3",
"Programming Language :: Python :: 3.12",
"Programming Language :: Python :: 3.13",
"Programming Language :: Python",
"Topic :: Scientific/Engineering",
]
Expand Down

0 comments on commit f596dbd

Please sign in to comment.