[Feature] - nice progress bar during downloading of SLC data

[cmarshak]

Is your feature request related to a problem? Please describe.
Downloading SLC data takes a long time. Would be nice to track.

Describe the solution you'd like
Something like this - https://gist.github.com/wy193777/0e2a4932e81afc6aa4c8f7a2984f34e2 - would be happy to make a pull request if the feature would be welcome. Understand if this is out of scope.

Describe alternatives you've considered
I think there are other progress bars, but am not familiar with them.

Additional context
There is a lot of output during downloading e.g.

response: 307
Redirect to https://sentinel1.asf.alaska.edu/SLC/SB/S1B_IW_SLC__1SDV_20210723T014947_20210723T015014_027915_0354B4_B3A9.zip
response: 302
Redirect to https://dy4owt9f80bz7.cloudfront.net/...
response: 200

Rather be able to track progress more clearly.

edit: realizing this feature request might be a bit hasty as there might be use cases which this is not desired and annoying - but hopefully there might be a smart way to integrate such a feature.

[jhkennedy]

@glshort we've added a tqdm process bar to the HyP3 SDK so you get pretty progress bars in Jupyter Notebooks and text ones everywhere else. E.g.,
https://github.com/ASFHyP3/hyp3-sdk/blob/develop/hyp3_sdk/util.py#L113-L121

One thing to be aware of, is that if you do want to use the Jupyter support, you'll need to be careful with the import b/c you can be running in a Jupyter kernel, but not have all the expected Jupyter dependencies, so we do this:
https://github.com/ASFHyP3/hyp3-sdk/blob/develop/hyp3_sdk/util.py#L54-L61

[carmine1990]

@jhkennedy using your link i customized the download.py script in the download folder so that, if the processes params is 1 (the customizeation doesn't support pool() method) now you can see the progres of the download.

The function to modify is asf_search/download/download.py:

def download_url(url: str, path: str, filename: str = None, session: ASFSession = None ) -> None:
"""
Downloads a product from the specified URL to the specified location and (optional) filename.
:param url: URL from which to download
:param path: Local path in which to save the product
:param filename: Optional filename to be used, extracted from the URL by default
:param session: The session to use, in most cases should be authenticated beforehand
:return:
"""
if filename is None:
filename = os.path.split(urllib.parse.urlparse(url).path)[1]
if not os.path.isdir(path):
raise ASFDownloadError(f'Error downloading {url}: directory not found: {path}')
if os.path.isfile(os.path.join(path, filename)):
warnings.warn(f'File already exists, skipping download: {os.path.join(path, filename)}')
return
if session is None:
session = ASFSession()
def strip_auth_if_aws(r, *args, **kwargs):
if 300 <= r.status_code <= 399 and 'amazonaws.com' in urllib.parse.urlparse(r.headers['location']).netloc:
location = r.headers['location']
r.headers.clear()
r.headers['location'] = location
response = session.get(url, stream=True, hooks={'response': strip_auth_if_aws})
response.raise_for_status()
with tqdm.wrapattr(open(os.path.join(path, filename),'wb'),
'write', miniters=1,
desc=filename,
total=int(response.headers.get('content-length', 0))) as f:
for chunk in response.iter_content(chunk_size=31457280):
f.write(chunk)

[scottstanie]

Is there still interest in an implementation of this? if nothing else, for the top-level loop that shows the total number of files to download would be only a one line change, and it's nice that now you can disable the progress bars with TQDM_DISABLE=1. Not sure if you'd want to add the dependency though.