Final Algorithm Development Point Release 4.1 (R4.1)

Updates since last release (Final Algorithm Development Release R4.0 v1.0.0):

• Increase a buffer declared that is used to check the ancillary input's coverage at the right side of the dateline from 1 arcsec (~0.00027777 deg ~= 30 m) to 33 arcsec (about 1 km);
• Add support to input ancillary files provided with longitude range [0, 360] degrees, rather than [-180, +180] degrees;
• Minor style changes: improved docstrings/comments, indentation fixes, etc.

Final Algorithm Development Release 4 (R4)

Updates since last release (Cal/Val Release 3.4 v0.5.3):

• Handle antimeridian crossing;
• Add high-aerosol class remapping;
• Adds support to no-/partial landcover coverage due to low/high latitudes, and save coverage status on the metadata fields: DEM_COVERAGE, LANDCOVER_COVERAGE, and WORLDCOVER_COVERAGE;
• Set default for apply_ocean_masking (in the default runconfig file) to False;
• Rename metadata fields imported from HLS product from SPATIAL_COVERAGE and CLOUD_COVERAGE to INPUT_HLS_PRODUCT_SPATIAL_COVERAGE and INPUT_HLS_PRODUCT_CLOUD_COVERAGE, respectively;
• Recompute and store the new metadata fields SPATIAL_COVERAGE and CLOUD_COVERAGE considering fill value and ocean masked data as invalid data, and add field SPATIAL_COVERAGE_EXCLUDING_MASKED_OCEAN that takes into account only the fill value (i.e., excludes ocean masked data);
• Save new metadata fields describing processing parameters used for generating the DSWx-HLS product: SHADOW_MASKING_ALGORITHM, MIN_SLOPE_ANGLE, MAX_SUN_LOCAL_INC_ANGLE, MASK_ADJACENT_TO_CLOUD_MODE, FOREST_MASK_LANDCOVER_CLASSES, OCEAN_MASKING_SHORELINE_DISTANCE_KM, AEROSOL_CLASS_REMAPPING_ENABLED and OCEAN_MASKING_ENABLED;
• Update the shade of the gray color to represent clouds to Red: 175, Green: 175, and Blue: 175;
• Add Copernicus Landcover class 50 (urban/built-up) to the runconfing forest_mask_landcover_classes default;
• Attempts to read the WorldCover dataset year from WorldCover metadata. If it fails, infer it form the WorldCover file description. If it still fails, set year offset to 0.

Cal/Val Point Release 3.4

Updates since last release (Cal/Val release 3.3 v0.5.2):

• Fix and improve the ancillary inputs checks;
• Removes unnecessary code;
• Removing my author's name and email from setup.py.

Cal/Val Point Release 3.3

Updates since last release (Cal/Val release v0.5.1):

• Ocean masking using a shoreline shape file (GSHHS) - new fields were added to the runconfig: shoreline_shape_file and shoreline_shape_file_description;
• Updated CONF layer from confidence levels 0-100 to level classes 0-4, 10-14 (cloud masked), 20-24 (snow masked).
• Cumulative reflectance no-data masking;
• Refactored code;
• Renamed runconfig parameters dem_description, landcover_description, and worldcover_description to dem_file_description, landcover_file_description, and worldcover_file_description, respectively;
• Created a check for ancillary inputs that can be enabled through the runconfig parameter check_ancillary_inputs_coverage. The check tests for the existence of the ancillary input file and that it covers totally the HLS/DSWx-HLS product;
• Update messages logged/printed to the user;
• Set runconfig variable exclude_psw_aggressive_in_browse default to True per OPERA Project request.

Cal/Val Point Release 3.2

Updates since last release (Cal/Val release v0.5):

• Updated browse image and color for Partial Surface Water;
• Fix edge issue (mask DSWx-HLS layers based on Fmask fill_value);
• Add an option to select Copernicus Land Cover 100m forest classes for land cover masking;
• Pull from Docker image python:3.9 rather than from image python:3.8;
• Add flag check_ancillary_inputs_coverage;
• Substitute flag flag_use_otsu_terrain_masking with parameter shadow_masking_algorithm.

Cal/Val Point Release 3.1

Updates since last release (Cal/Val release v0.4):

• Added code to generate a browse image as part of the DSWx-HLS product using the WTR layer and saving it in the PNG format. The image dimensions can be specified through the runconfig file.
• Implemented a new shadow masking algorithm based on the sun's local incidence angle and set it as the default algorithm for shadow masking. Provided the option to use the previous algorithm based on Otsu thresholding.
• Created snow class from input HLS product Q/A (FMask) layer. The new class was added to WTR and CLOUD layers.
• Refactored code. The new code runs faster, and it is easier to understand.
• Improved metadata following findings in SDS E2E tests.
• Fixed a critical land cover bug and improved land cover masking.
• Clip negative reflectance.
• Updated wigt threshold from 0.0124 to 0.124.
• Added three options to handle areas adjacent to cloud/cloud-shadow: "mask" (default), "cover", and "ignore".

Cal/Val release

Updates since last release (Mid-May 2022 release v0.3):

• The DEM and land cover maps (Copernicus Land Cover 100m and ESA Worldcover 10m) are now used by the SAS to compute the DSWx-HLS LAND layer and mask the interpreted layer WTR-1 to WTR-1;
• Removed built-up layer as an input dataset;
• Updated runconfig schema and defaults to include the world cover map and removed the built-up layer field.
  Improved run time due to new algorithms used to compute diagnostic tests and interpreted layer;
• Updated output DSWx-HLS product color tables and classes to match the Product Specification Document for Dynamic Surface Water Extent from Harmonized Landsat and Sentinel-2;
• Docker image can run with local user instead of conda:conda.

Mid-May 2022 release

Updates since last delivery (beta v0.2):

• Add error tolerance to function compare_dswx_hls_products().
• Fix bug printing wrong layer name in function compare_dswx_hls_products().
• Remove the creation of directories input_dir, scratch_dir, runconfig, and scratch_dir inside Docker container.
• Fix value assignment of DSWx-HLS products metadata SENSOR_PRODUCT_ID.
• Update comments on DSWx-HLS runconfig schema and default files.
• Update the computation of the joint land cover map from ESA Landcover 100m and ESA Worldcover 10m.
• Add code that masks the DSWx-HLS interpreted layer using the joint land cover map.

BETA: Beta Delivery

The main executable is the Python script dswx_hls.py that computes the L3_DSWx_HLS product from the input HLS product. The script supports HLS products versions 1.4 and 2.0. However, only version 2.0 is expected to be used in production. The script accepts parameters from the command-line interface (CLI) or from a runconfig file written in the YAML format.

This release generates the following layers:

Water classification (WTR)
Binary Water (BWTR)
Diagnostic layer (DIAG)
Interpretation of diagnostic layer into water classes (WTR-1)
Interpreted layer refined using land cover and terrain shadow testing (WTR-2)
Cloud/cloud-shadow classification (CLOUD)
Terrain shadow layer (SHAD)
Digital elevation model (DEM)

The current release does not generate the following layers:

Confidence layer (CONF)
Land cover classification (LAND)

The layers that are not computed are saved in the L3_DSWx_HLS product as zero arrays.

The software also allows for saving each layer into separate GeoTIFF files, enabling the use of color tables which is not allowed in multi-band GeoTIFF files.