Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue in creating LISFLOOD input files for Tapi basin in India #153

Open
diksha922 opened this issue Mar 3, 2024 · 23 comments
Open

Issue in creating LISFLOOD input files for Tapi basin in India #153

diksha922 opened this issue Mar 3, 2024 · 23 comments

Comments

@diksha922
Copy link

Sir/ Madam,

I want to use LISFLOOD model for hydrological modelling over Tapi basin in India, using the rainfall data from Indian Meteorological Department at 0.25 degree and temperature at 1 degree, on a daily time step. However, I am facing issues in creating the meteorological inputs files using these inputs.

Further, I am not getting how to create the other input files such as topography, landuse, soil map etc. required for running the hydrological model over Tapi basin.

Please help me in creating the input files for the study area for running the hydrological model.

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

thank you for your interest in using OS LISFLOOD for hydrological modelling in the Tapi basin.

OS LISFLOOD requires the following meteorological inputs: total precipitation, average temperature, reference evapotranspiration.

All the input meteorological forcings and static maps (topography, land use, soil maps, ...) must have the same spatial extent, the same reference system, the same pixel size. This is a strict requirement for model implementation: more details can be found here https://ec-jrc.github.io/lisflood-code/4_Static-Maps-introduction/

Therefore, the first step for the preparation of the input meteorological forcings and of static maps consists in defining spatial extent, reference system, and pixel size for your application.

Your rainfall data and temperature data have different pixel size, and they require re-gridding prior to their use within OS LISFLOOD. Depending on the format and structure (e.g. GRIB) of your rainfall and temperature dataset, you might be interested in checking the functionalities of the open-source tool pyg2p: https://github.com/ec-jrc/pyg2p

As a follow up of this previous conversation ec-jrc/lisflood-lisvap#56, we shared with you the mask map and the ERA5 meteorological variables for the computation of reference evapotranspiration values for the Tapi basin using OS LISVAP. The mask map and all the meteorological variables had 3arcmin (or 0.05 degrees) resolution.

In case you decided to use 3arcmin (or 0.05 degrees) resolution for your OS LISFLOOD model set up, you might be interested in the openly available LISFLOOD static maps dataset at 3arcmin (or 0.05 degrees) resolution, for the global domain: https://data.jrc.ec.europa.eu/dataset/68050d73-9c06-499c-a441-dc5053cb0c86. This manuscript https://egusphere.copernicus.org/preprints/2023/egusphere-2023-1306/ provides a detailed description of the raw datasets and of the protocols used for the generation of each map of the openly available dataset.

Clearly, users can generate their own input maps dataset: the technical requirements and the use of each static map within LISFLOOD are explained here: https://ec-jrc.github.io/lisflood-code/4_Static-Maps-introduction/.

We hope that our answer helps,
please do not hesitate to get in touch for further questions,
kind regards,
on behalf of the developers team,
Stefania

@diksha922
Copy link
Author

diksha922 commented Mar 5, 2024

Mam,

I have generated the potential evapotranspiration using the input file provided at: ec-jrc/lisflood-lisvap#56, at 3 arc resolution. I have rainfall available at 0.25 degree and temperature at 1 degree.

I am not getting how to convert all the input files to same resolution. Please help me for getting the same, may be some reference examples may help.

Also, it will be helpful if some guide/ reference may be provided to generate LISFLOOD compatible inputs from .mat file of rainfall and temperature.

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

what is the format of your rainfall and temperature dataset? Different solutions are possible depending on the format of your datasets.

Kind regards,
on behalf of the developers team,
Stefania

@diksha922
Copy link
Author

Madam,

The rainfall data is available as gridded dataset having three dimensions (X, Y, value) with 0.25 degree for rainfall and 1 degree for temperature, in .mat format.

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

thanks for providing more details.

Unfortunately, we cannot provide support for the conversion and re-gridding of matlab files.

However, online blogs propose various solutions. A quick internet search allowed to find a few examples: https://it.mathworks.com/matlabcentral/answers/565163-write-a-mat-file-into-netcdf-file and https://gdal.org/index.html. Please note that these are only examples. These are neither recommendations nor guidelines. We do not work with .mat files in our operational protocols.

When defining your working protocol, you should use the template map used in LISVAP to guarantee the consistency between LISFLOOD forcings files.

Kind regards,
on behalf of the developers team,
Stefania

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi , Can you please help to get ERA5 daily precipitation data for Tapi basin at 3 arc resolution, for the duration from 1979 onwards?

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

ERA5 is an openly available reanalysis dataset, you can find more information about the dataset and opportunities for download here: https://www.ecmwf.int/en/forecasts/dataset/ecmwf-reanalysis-v5

To re-grid ERA5 data from native resolution to 3 arcmin resolution you can use the open source tool https://github.com/ec-jrc/pyg2p.

We can surely share with you the cut-out of the ERA5-derived 3 arcmin total precipitation maps from 01/01/1979 to 31/12/2019. We will extend the 3 arcmin gridded dataset to include the recent years in the next months.

Please let us know whether the currently available dataset suits your purposes.

Kind regards,
On behalf of the developers team,
Stefania

@diksha922
Copy link
Author

diksha922 commented Jun 28, 2024 via email

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922 ,

you should have received an email from JRCbox: the email contains the link to download the data.
The password is ERA5Tapi!

According to OS LISLFOOD time convention (https://ec-jrc.github.io/lisflood-code/2_ESSENTIAL_time-management/), the time stamp of the daily total precipitation values is the end of the interval: for example, daily total precipitation of 01 Jan 1979 has time stamp 02 Jan 1979 00:00.

Please do not hesitate to let me know whether you have problems with the link or the data.

Kind regards,
On behalf of the developers team,
Stefania

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi.

Thank you for providing the dataset.

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi .

I have prepared the set up for Tapi basin, but I am getting the errors and unable to run the hydrological model. The copy of the errors are attached herewith.

I have also attached the settings file for Tapi basin.

Can you please help in this?

Error.odt
prerun.txt

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

the errors are due to misconfigured paths or missing paths in the settings .xml.

The error message states "Missing files or misconfigured paths to run LisFlood, according activated modules. Please check your settings file /home/diksha/Documents/LISFLOOD/lisflood-code/tests/data/tapi/settings/prerun.xml."

In order to be able to run the model, you will need to solve each one of the errors reported by the message.

I recommend to focus on one error at a time: implement the required changes, and verify whether that error message disappears.

For example, "setting ForestFraction refers to a non existing path or a not well-formed float value: /home/diksha/Documents/LISFLOOD/lisflood-code/tests/data/tapi/settings/../maps/landuse/fracforest_03min"
The model is not able to find the map fracforest_03min, according to the settings file, the model is using the path "/home/diksha/Documents/LISFLOOD/lisflood-code/tests/data/tapi/settings/../maps/landuse/".

Furthermore, I had a quick look at the settings xml that you kindly shared. Did you delete some of the lines of https://github.com/ec-jrc/lisflood-code/blob/master/src/lisfloodSettings_reference.xml ? I do not seem able to find the paths to the LAI maps (e.g. https://github.com/ec-jrc/lisflood-code/blob/master/src/lisfloodSettings_reference.xml#L935C16-L935C30).

I hope that my answer helps,
Kind regards,
On behalf of the developers team,
Stefania

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi,

I am still facing issues in creating the settings file. In the given test cases, LISFLOOD is being used for simulating nutrient concentration etc., which requires more inputs.

Can you please share some reference settings files where the LISFLOOD model is used for hydrological modeling and simulating streamflow using meteorological data?

Thanks & Regards,

Diksha Gupta

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi

There are number of settings and input maps mentioned in the user manual to run the LISFLOOD model. It will be good if you could provide the reference settings file which will let me know the mandatory requirement for running LISFLOOD for flood simulations i.e. hydrological modelling purpose only.

Thanks.

@StefaniaGrimaldi
Copy link
Contributor

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi.

Thank you for guiding me to run the basic LISFLOOD model.

Now, I am trying to run the lisflood calibration for the Tapi basin to calibrate the basin with the observed streamflow at the gauging site.

I am following the steps mentioned on the web to install and run lisflood calibration. However, I am unable to run the calibration tool for the basin.

The steps mentioned in the portal are not clear to me. Even I am not getting the sequence of steps to run the lisflood calibration.

Is there a requirement to create an environment?

Please guide me to understand the step-by-step process.

Thanks & Regards,

Diksha Gupta

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

please find our answer here ec-jrc/lisflood-calibration#13

Kind regards,
Stefania

@diksha922
Copy link
Author

diksha922 commented Jul 27, 2024

Hello @StefaniaGrimaldi.

Thank you for guiding me to run the basic LISFLOOD model.

Now, I am trying to run the lisflood calibration for the Tapi basin to calibrate the basin with the observed streamflow at the gauging site.

I am following the steps mentioned on the web to install and run lisflood calibration. However, I am unable to run the calibration tool for the basin.

The steps mentioned in the portal are not clear to me. Even I am not getting the sequence of steps to run the lisflood calibration.

Is there a requirement to create an environment?

Please guide me to understand the step-by-step process.

Thanks & Regards,

Diksha Gupta

Hello Stefania.

On running the basic LISFLOOD model for the Tapi basin with the Global Parameter file, I am getting the discharge series with the values of order 1e31 for one outlet and 1e-5 for other outlet, which is infeasible (output attached)
discharge.zip
.

I have used all the inputs from ERA5 at daily time step, shared with me.

I have checked multiple times for the error but unable to figure out the reason.

I have attached herewith the settings files I have used for the model.
settings_tapi-PreRun.txt

Can you please help me in this?

Thanks & Regards,

Diksha Gupta

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

thanks for reaching our and for sharing detailed information about your problem.

The diss.tss file contains the following IDs: 4501, 4559, 4685

ID 4501 is outside of your mask map. OS LISFLOOD does not compute values outside of the mask map. Therefore, the time series is composed by NaN (1e31).

The screenshot below shows in yellow the mask map, in shades of blue the 0.05 degrees upstream area, in red the IDs of the outlet map. ID 4501 is highlighted by the red circle.
This happens because the outlets map is the cut-out of the global outlets map.
You can ignore ID 4501, or you can generate your own list of outlets.

To generate your own list of outlets, you have two options:
(1) produce your own outlets map;
(2) type the coordinates of the desired points in the .xml settings file: https://github.com/ec-jrc/lisflood-code/blob/master/src/lisfloodSettings_reference.xml#L219 (you can find a compiled example here: https://github.com/ec-jrc/lisflood-usecases/blob/master/LF_ETRS89_UseCase/settings_ETRS89-Run.xml#L177, please note that you can add several points)

image

Regarding the small values of the other time series, could you please verify that your settings .xml?
When running OS LISFLOOD using a geographic coordinate system, it is necessary to activate "gridSizeUserDefined" and make use of the pixArea map: https://github.com/ec-jrc/lisflood-code/blob/master/src/lisfloodSettings_reference.xml#L33;
https://github.com/ec-jrc/lisflood-code/blob/master/src/lisfloodSettings_reference.xml#L4481

I also noticed the following setting:

The CalendarDayStart is the first time stamp of the meteorological forcings. If you are using the files previously shared with you, the CalendarDayStart is 02/01/1979 00:00.

Finally, I would like to mention that it is recommended to use all available forcings to complete the PreRun: https://ec-jrc.github.io/lisflood-code/3_step5_model-initialisation/

I hope this answer helps,
kind regards,
on behalf of the developers team,
Stefania

@diksha922
Copy link
Author

Hello Stefania.

Thank you for the information, it helps to generate streamflow from basic LISFLOOD model.

I am finding issues in running the calibration of the LISFLOOD model.

There are some inputs and outputs provided in the documentation of LISFLOOD calibration. However, the complete calibration setup is not provided.

It will be good if you could provide the complete reference example setup for running the calibration of LISFLOOD model. This will greatly help in running the calibration for my basin.

Please help me in this.

Thanks,

Diksha Gupta

@diksha922
Copy link
Author

Hello Stefania.

Thank you for the information, it helps to generate streamflow from basic LISFLOOD model.

I am finding issues in running the calibration of the LISFLOOD model.

There are some inputs and outputs provided in the documentation of LISFLOOD calibration. However, the complete calibration setup is not provided.

It will be good if you could provide the complete reference example setup for running the calibration of LISFLOOD model. This will greatly help in running the calibration for my basin.

Please help me in this.

Thanks,

Diksha Gupta

Hello @StefaniaGrimaldi .

Any updates on the above query?

Thanks & Regards,

Diksha Gupta

@StefaniaGrimaldi
Copy link
Contributor

Dear @diksha922,

we are pleased to know about your progress with the OS LISFLOOD stand-alone prerun and run.
It is strictly necessary to ensure the correctness of settings and set-up of the stand-alone model prerun and run before calibrating the parameters.

Concerning your latest enquiry, please refer to our answer here ec-jrc/lisflood-calibration#13 (you can find a draft of the new documentation and several answers to enquiries from other users).
An example of .xml settings for the global domain is available here https://github.com/ec-jrc/lisflood-calibration/blob/master/integration/settings_GloFAS3arcmin_referencetemplate.xml

We will do our best to reply to specific questions/issues: please share with us the details of your doubt or the error log.

Kind regards,
on behalf of the developers team,
Stefania

@diksha922
Copy link
Author

Hello @StefaniaGrimaldi,

I am trying to create the calibration files for Tapi Basin. However, I am unable to follow the steps of LISFLOOD Calibration.

I am not able to create "station_data.csv", "station_link.csv" file. Also, kindly let me know the step-by-step procedure to run calibration for my basin.

Thanks & Regards,

Diksha Gupta

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants