Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve the Memory Efficiency of Water Masking #280

Open
wants to merge 6 commits into
base: develop
Choose a base branch
from

Conversation

AndrewPlayer3
Copy link
Contributor

@AndrewPlayer3 AndrewPlayer3 commented Mar 7, 2025

This PR modifies water_mask.py such that create_water_mask is able to utilize the COG features of the water mask files in AWS. The improvement comes from subsetting the water mask tiles to the projection window of the input image. This means that GDAL only has to work with a small portion of the data, rather than the multiple GBs that the full water masks may take when uncompressed.

This should address ASFHyP3/hyp3#2640

@AndrewPlayer3 AndrewPlayer3 added the patch Bump the patch version number of this project label Mar 7, 2025
@AndrewPlayer3 AndrewPlayer3 requested review from a team as code owners March 7, 2025 21:28
Copy link
Contributor

github-actions bot commented Mar 7, 2025

Coverage

Coverage Report
FileStmtsMissCoverMissing
/home/runner/micromamba/envs/hyp3-isce2/lib/python3.11/site-packages/hyp3_isce2
   __main__.py26292%41, 55
   burst.py2645778%94, 104, 117–120, 133–138, 152–165, 183–190, 208–218, 286, 334–342, 362–376, 614–616, 642
   dem.py53394%103–112
   insar_stripmap.py755033%37–100, 104–106, 120–137
   insar_tops.py825533%52–117, 147–200, 225–243
   insar_tops_burst.py1339727%65–151, 162–215, 226–247, 251–253, 284–317
   logger.py4175%9
   merge_tops_bursts.py6319685%186, 203, 272, 278, 335, 351, 359, 394, 398, 486, 513, 591, 631–654, 702–717, 741, 746, 805, 834, 978, 1161–1181, 1200–1217, 1240–1273, 1284–1292, 1302–1322, 1326
   packaging.py19814825%44–46, 72–96, 144–242, 261–277, 296–306, 318–346, 358–360, 389–478, 482–488
   s1_auxcal.py23291%54, 62
   slc.py502354%24–31, 35–40, 51–63
   stripmapapp_alos.py49786%102, 143, 146–151
   topsapp.py70889%78, 108, 130–136
   utils.py1911095%107–118, 123, 201, 223–225, 232, 389, 434
   water_mask.py59493%102, 112–114
TOTAL192956371% 

Tests Skipped Failures Errors Time
85 0 💤 0 ❌ 0 🔥 34.787s ⏱️

@AndrewPlayer3 AndrewPlayer3 marked this pull request as draft March 7, 2025 21:50
@AndrewPlayer3 AndrewPlayer3 marked this pull request as ready for review March 7, 2025 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
patch Bump the patch version number of this project
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant