Problem using UMEP ERA5 Meterological downloading tool

[croostanse]

I am trying to use the UMEP Meterological Data: Download Data (ERA5).

I have created an account to be aple to use the API to download the metrological data.
Qgis has been reinstalled and I have tried both the 3.22 version and 3.28 (latest long term release).
I am using pip (python 3.9)
SuPy version: 2022.9.22

These are the setting I have used

After putting in this information I got this error message and I am unable to get any metrological data.
QGIS version: 3.22.4-Białowieża
QGIS code revision: ce8e65e9
Qt version: 5.15.2
Python version: 3.9.5
GDAL version: 3.4.1
GEOS version: 3.10.2-CAPI-1.16.0
PROJ version: Rel. 8.2.1, January 1st, 2022
PDAL version: 2.3.0 (git-version: 9f35b7)
Algorithm started at: 2023-06-01T15:24:31
Algorithm 'Meteorological Data: Download data (ERA5)' starting…
Input parameters:
{ 'CRS' : QgsCoordinateReferenceSystem('EPSG:3009'), 'DATEINIEND' : '2019-01-16T00:00:00', 'DATEINISTART' : '2019-01-01T00:00:00', 'DIAG_HEIGHT' : 100, 'INPUT_POINT' : '117850.660703,6669733.798057 [EPSG:3009]', 'OUTPUT_DIR' : 'TEMPORARY_OUTPUT' }

SuPy version: 2022.9.22
INPUT PARAMETERS:
lat = 60.13930272777642
lon = 14.421407895689653
Start = 2019-01-01T00:00:00
End = 2019-01-16T00:00:00
Diagnostic height = 100.0
Output folder = C:/Users/caro0725/AppData/Local/Temp/processing_REsXqt/b32f49e7cab54c759f422fa1b068b5b2/OUTPUT_DIR
Traceback (most recent call last):
File "C:\Users/caro0725/AppData/Roaming/QGIS/QGIS3\profiles\Engelska/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 366, in retrieve
result.download(target)
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 171, in download
return self._download(self.location, self.content_length, target)
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 121, in _download
with tqdm(
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 1095, in init
self.refresh(lock_args=self.lock_args)
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 1344, in refresh
self.display()
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 1492, in display
self.sp(self.str() if msg is None else msg)
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 347, in print_status
fp_write('\r' + s + (' ' * max(last_len[0] - len_s, 0)))
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 340, in fp_write
fp.write(str(s))
File "C:\Users\caro0725\AppData\Roaming\Python\Python39\site-packages\tqdm\utils.py", line 71, in getattr
return getattr(self._wrapped, name)
AttributeError: 'NoneType' object has no attribute 'write'

Execution failed after 259.75 seconds (4 minutes 20 seconds)

Loading resulting layers
Algorithm 'Meteorological Data: Download data (ERA5)' finished

Any body knows what could be wrong?
Am I missing some python library?

[biglimp]

Open the Python console in QGIS and run the same tool again. Then you get the log-ingo from the cdsapi. Then you get some more info on what might have gone wrong.

[croostanse]

I dont know why. But when I tried running it on qgis version 2.22 it now works. Thank you for your answer.

[phkam1]

I encounter the same error too.
QGIS version: 3.30.2
OS: Windows 10
Python version: 3.9.5
SuPy version: 2022.9.22

This is the output from Python Console:

Python Console

Use iface to access QGIS API interface or type help(iface) for more info

Security warning: typing commands from an untrusted source can harm your computer

--- Logging error ---
Traceback (most recent call last):
File "C:\PROGRA1\QGIS331.2\apps\Python39\lib\logging_init_.py", line 1086, in emit
stream.write(msg + self.terminator)
AttributeError: 'NoneType' object has no attribute 'write'
Call stack:
File "C:\Users/phkam/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 364, in retrieve
result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 420, in _api
self._status(url)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 408, in status
self.info("%s", i)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 530, in info
self.logger.info(*args, **kwargs)
Message: '%s'
Arguments: ('Welcome to the CDS',)
--- Logging error ---
Traceback (most recent call last):
File "C:\PROGRA1\QGIS331.2\apps\Python39\lib\logging_init.py", line 1086, in emit
stream.write(msg + self.terminator)
AttributeError: 'NoneType' object has no attribute 'write'
Call stack:
File "C:\Users/phkam/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 364, in retrieve
result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 424, in api
self.info("Sending request to %s", url)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 530, in info
self.logger.info(*args, **kwargs)
Message: 'Sending request to %s'
Arguments: ('https://cds.climate.copernicus.eu/api/v2/resources/reanalysis-era5-single-levels',)
--- Logging error ---
Traceback (most recent call last):
File "C:\PROGRA1\QGIS331.2\apps\Python39\lib\logging_init.py", line 1086, in emit
stream.write(msg + self.terminator)
AttributeError: 'NoneType' object has no attribute 'write'
Call stack:
File "C:\Users/phkam/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 364, in retrieve
result = self._api("%s/resources/%s" % (self.url, name), request, "POST")
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 477, in api
self.info("Request is %s" % (reply["state"],))
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 530, in info
self.logger.info(*args, **kwargs)
Message: 'Request is completed'
Arguments: ()
--- Logging error ---
Traceback (most recent call last):
File "C:\PROGRA1\QGIS331.2\apps\Python39\lib\logging_init.py", line 1086, in emit
stream.write(msg + self.terminator)
AttributeError: 'NoneType' object has no attribute 'write'
Call stack:
File "C:\Users/phkam/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 366, in retrieve
result.download(target)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 171, in download
return self._download(self.location, self.content_length, target)
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 101, in _download
self.info("Downloading %s to %s (%s)", url, target, bytes_to_string(size))
File "C:\Users\phkam\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 530, in info
self.logger.info(*args, **kwargs)
Message: 'Downloading %s to %s (%s)'
Arguments: ('https://download-0010-clone.copernicus-climate.eu/cache-compute-0010/cache/data5/adaptor.mars.internal-1685652756.8911376-10066-4-34d58a53-bea9-47e8-8b8a-91a00392e24b.nc', '51.0N1.0W-200001-sfc.nc', '6.8K')

[biglimp]

@sunt05 , any ideas?

[sunt05]

Sorry - no idea at all - could be an upstream issue as I see it's from CDSAPI/tqdm.

[phkam1]

Thankyou for looking at it. I have switched to using CDSAPI (https://cds.climate.copernicus.eu/api-how-to) meanwhile.

[croostanse]

For me it seems to have helped to update python libraries such as pip. I had a problem with it not being properly installed. I did that and thenit worked after I restarted my computer. I dont know if that can be of any help.

[croostanse]

I use the API as well

[Witellsteim]

Hello all, i get a similar Error when trying to run the ERA5 downloader:

QGIS version: 3.22.10-Białowieża
QGIS code revision: 6a63fd37
Qt version: 5.15.3
Python version: 3.9.5
GDAL version: 3.5.1
GEOS version: 3.10.3-CAPI-1.16.1
PROJ version: Rel. 9.0.1, June 15th, 2022
PDAL version: 2.4.2 (git-version: 6cb4e7)
Algorithm started at: 2023-06-14T14:40:52
Algorithm 'Meteorological Data: Download data (ERA5)' starting…
Input parameters:
{ 'CRS' : QgsCoordinateReferenceSystem('EPSG:25833'), 'DATEINIEND' : '2019-07-24T00:00:00', 'DATEINISTART' : '2019-07-23T00:00:00', 'DIAG_HEIGHT' : 100, 'INPUT_POINT' : '365616.952248,5807619.053826 [EPSG:25833]', 'OUTPUT_DIR' : 'TEMPORARY_OUTPUT' }

SuPy version: 2023.5.26.dev0
INPUT PARAMETERS:
lat = 52.402287657033995
lon = 13.02465110441006
Start = 2019-07-23T00:00:00
End = 2019-07-24T00:00:00
Diagnostic height = 100.0
Output folder = C:/Users/mabi/AppData/Local/Temp/processing_jrdFST/44ba8b82f3be4a278fdb7c154385669a/OUTPUT_DIR
Traceback (most recent call last):
File "C:\Users/mabi/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\processing_umep\preprocessor\copernicusera5_algorithm.py", line 182, in processAlgorithm
sp.util.gen_forcing_era5(lat, lon, startDate, endDate, hgt_agl_diag=diagHeight, dir_save=Path(outputDir))
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 733, in gen_forcing_era5
list_fn_sfc, list_fn_ml = load_filelist_era5(
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 627, in load_filelist_era5
download_era5(
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 523, in download_era5
download_cds(path_dir_save / fn_sfc, dict_req)
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\supy\util_era5.py", line 455, in download_cds
c.retrieve(**dict_req)
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 350, in retrieve
result.download(target)
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 173, in download
return self._download(self.location, self.content_length, target)
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\cdsapi\api.py", line 123, in _download
with tqdm(
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 1109, in init
self.refresh(lock_args=self.lock_args)
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 1361, in refresh
self.display()
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 1509, in display
self.sp(self.str() if msg is None else msg)
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 350, in print_status
fp_write('\r' + s + (' ' * max(last_len[0] - len_s, 0)))
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\tqdm\std.py", line 343, in fp_write
fp.write(_unicode(s))
File "C:\Users\mabi\AppData\Roaming\Python\Python39\site-packages\tqdm\utils.py", line 89, in getattr
return getattr(self._wrapped, name)
AttributeError: 'NoneType' object has no attribute 'write'

Execution failed after 1.03 seconds

Loading resulting layers
Algorithm 'Meteorological Data: Download data (ERA5)' finished

For now i am also using the CDS API but for convenience sake i hope this error gets resolved soon.

Cheers all,
W.

EDIT: After running it again directly after this try it is stuck in the process, and i cannot cancel the process, it continues running without producing results.

[biglimp]

I cannot replicate your error.

What you can do is to open the Python console in QGIS to see the log-info from cdsapi. Then you get some more info on what is happening. I can also recommend to visit https://www.shinyweatherdata.com/ where ERA5 data has been preprocessed from spatial to temporal dataset and thus the downloading goes much faster.