Skip to content

Commit

Permalink
Merge pull request #62 from IMMM-SFA/feature/improved_reservoir_schedule
Browse files Browse the repository at this point in the history
data driven reservoir behavior
  • Loading branch information
thurber authored Oct 28, 2021
2 parents 00f8e6c + c8c762a commit 12b5b5c
Show file tree
Hide file tree
Showing 39 changed files with 3,641 additions and 557 deletions.
222 changes: 89 additions & 133 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
## mosartwmpy

`mosartwmpy` is a python translation of MOSART-WM, a model for water routing and reservoir management written in Fortran. The original code can be found at [IWMM](https://github.com/IMMM-SFA/iwmm) and [E3SM](https://github.com/E3SM-Project/E3SM), in which MOSART is the river routing component of a larger suite of earth-science models. The motivation for rewriting is largely for developer convenience -- running, debugging, and adding new capabilities were becoming increasingly difficult due to the complexity of the codebase and lack of familiarity with Fortran. This version aims to be intuitive, lightweight, and well documented, while still being highly interoperable.
For a quick start, check out the [Jupyter notebook tutorial](https://github.com/IMMM-SFA/mosartwmpy/blob/main/notebooks/tutorial.ipynb)!

## getting started

Expand Down Expand Up @@ -49,7 +50,15 @@ Settings are defined by the merger of the `mosartwmpy/config_defaults.yaml` and
> read_from_file: true
> path: ./input/demand/demand_1981_05.nc
> reservoirs:
> path: ./input/reservoirs/reservoirs.nc
> enable_istarf: true
> parameters:
> path: ./input/reservoirs/reservoirs.nc
> dependencies:
> path: ./input/reservoirs/dependency_database.parquet
> streamflow:
> path: ./input/reservoirs/mean_monthly_reservoir_flow.parquet
> demand:
> path: ./input/reservoirs/mean_monthly_reservoir_demand.parquet
> ```
`mosartwmpy` implements the [Basic Model Interface](https://csdms.colorado.edu/wiki/BMI) defined by the CSDMS, so driving it should be familiar to those accustomed to the BMI. To launch the simulation, open a python shell and run the following:
Expand All @@ -58,7 +67,7 @@ Settings are defined by the merger of the `mosartwmpy/config_defaults.yaml` and
from mosartwmpy import Model
# path to the configuration yaml file
config_file = "config.yaml"
config_file = 'config.yaml'
# initialize the model
mosart_wm = Model()
Expand All @@ -71,161 +80,108 @@ mosart_wm.update()
mosart_wm.update_until(mosart_wm.get_end_time())
```
Alternatively, one can update the settings via code in the driving script using dot notation:
## model input

```python
from mosartwmpy import Model
from datetime import datetime
Input for `mosartwmpy` consists of many files defining the characteristics of the discrete grid, the river network, surface and subsurface runoff, water demand, and dams/reservoirs.
Currently, the gridded data is expected to be provided at the same spatial resolution.
Runoff input can be provided at any time resolution; each timestep will select the runoff at the closest time in the past.
Currently, demand input is read monthly but will also pad to the closest time in the past.
Efforts are under way for more robust demand handling.
Dams/reservoirs require four different input files: the physical characteristics, the average monthly flow expected during the simulation period, the average monthly demand expected during the simulation period, and a database mapping each GRanD ID to grid cell IDs allowed to extract water from it.
These dam/reservoir input files can be generated from raw GRanD data, raw elevation data, and raw ISTARF data using the [provided utility](mosartwmpy/utilities/CREATE_GRAND_PARAMETERS.md).
The best way to understand the expected format of the input files is to examine the sample inputs provided by the download utility: `python -m mosartwmpy.download`.

mosart_wm = Model()
mosart_wm.initialize()

mosart_wm.config['simulation.name'] = 'Tutorial'
mosart_wm.config['simulation.start_date'] = datetime(1981, 5, 24)
mosart_wm.config['simulation.end_date'] = datetime(1985, 5, 26)
# etc...
```
## model output

One can use the usual python plotting libraries to visualize data. Model state and output are stored as one-dimensional numpy ndarrays, so they must be reshaped to visualize two-dimensionally:
By default, key model variables are output on a monthly basis at a daily averaged resolution to `./output/<simulation name>/<simulation name>_<year>_<month>.nc`. See the configuration file for examples of how to modify the outputs, and the `./mosartwmpy/state/state.py` file for state variable names.

Alternatively, certain model outputs deemed most important can be accessed using the BMI interface methods. For example:
```python
import xarray as xr
import matplotlib.pyplot as plt
from mosartwmpy import Model

mosart_wm = Model()
mosart_wm.initialize('./config.yaml')
mosart_wm.initialize()

mosart_wm.update_until(mosart_wm.get_end_time())
# get a list of model output variables
mosart_wm.get_output_var_names()

surface_water = mosart_wm.get_value_ptr('surface_water_amount')
# get the flattened numpy.ndarray of values for an output variable
supply = mosart_wm.get_value_ptr('supply_water_amount')
```

# create an xarray from the data, which has some convenience wrappers for matplotlib methods
data_array = xr.DataArray(
surface_water.reshape(mosart_wm.get_grid_shape()),
dims=['latitude', 'longitude'],
coords={'latitude': mosart_wm.get_grid_x(), 'longitude': mosart_wm.get_grid_y()},
name='Surface Water Amount',
attrs={'units': mosart_wm.get_var_units('surface_water_amount')}
)
## visualization

# plot as a pcolormesh
data_array.plot(robust=True, levels=32, cmap='winter_r')
`Model` instances can plot the current value of certain input and output variables (those available from `Model.get_output_var_name` and `Model.get_input_var_names`):

plt.show()
```python
from mosartwmpy import Model
config_file = 'config.yaml'
mosart_wm = Model()
mosart_wm.initialize(config_file)
for _ in range(8):
mosart_wm.update()

mosart_wm.plot_variable('outgoing_water_volume_transport_along_river_channel', log_scale=True)
```
![River transport](docs/_static/river_transport.png)

## model coupling

A common use case for `mosartwmpy` is to run coupled with output from the Community Land Model (CLM). To see an example of how to drive `mosartwmpy` with runoff from a coupled model, check out the [Jupyter notebook tutorial](https://github.com/IMMM-SFA/mosartwmpy/blob/main/notebooks/tutorial.ipynb)!

## model input
Using provided utility functions, the output of a simulation can be plotted as well.

Several input files in NetCDF format are required to successfully run a simulation, which are not shipped with this repository due to their large size. The grid files, reservoir files, and a small range of runoff and demand input files can be obtained using the download utility by running `python -m mosartwmpy.download` and choosing option 1 for "sample_input". Currently, all input files are assumed to be at the same resolution (for the sample files this is 1/8 degree over the CONUS). Below is a summary of the various input files:

<table>
<thead>
<tr>
<th>
Name
</th>
<th>
Description
</th>
<th>
Configuration Path
</th>
<th>
Notes
</th>
</tr>
</thead>
<tbody>
<tr>
<td>
Grid
</td>
<td>
Spatial constants dimensioned by latitude and longitude relating to the physical properties of the river channels
</td>
<td>
<code>grid.path</code>
</td>
<td/>
</tr>
<tr>
<td>
Land Fraction
</td>
<td>
Fraction of grid cell that is land (as opposed to i.e. ocean water) dimensioned by latitude and longitude
</td>
<td>
<code>grid.land.path</code>
</td>
<td>
As a TODO item, this variable should be merged into the grid file (historically it was separate for the coupled land model)
</td>
</tr>
<tr>
<td>
Reservoirs
</td>
<td>
Locations of reservoirs (possibly aggregated) and their physical and political properties
</td>
<td>
<code>water_management.reservoirs.path</code>
</td>
<td/>
</tr>
<tr>
<td>
Runoff
</td>
<td>
Surface runoff, subsurface runoff, and wetland runoff per grid cell averaged per unit of time; used to drive the river routing
</td>
<td>
<code>runoff.path</code>
</td>
<td/>
</tr>
<tr>
<td>
Demand
</td>
<td>
Water demand of grid cells averaged per unit of time; currently assumed to be monthly
</td>
<td>
<code>water_management.reservoirs.demand</code>
</td>
<td>
There are plans to support other time scales, such as epiweeks
</td>
</tr>
</tbody>
</table>

## model output
Plot the storage, inflow, and outflow of a particular GRanD dam:
```python
from mosartwmpy import Model
from mosartwmpy.plotting.plot import plot_reservoir
config_file = 'config.yaml'
mosart_wm = Model()
mosart_wm.initialize(config_file)
mosart_wm.update_until()

By default, key model variables are output on a monthly basis at a daily averaged resolution to `./output/<simulation name>/<simulation name>_<year>_<month>.nc`. See the configuration file for examples of how to modify the outputs, and the `./mosartwmpy/state/state.py` file for state variable names.
plot_reservoir(
model=mosart_wm,
grand_id=310,
start='1981-05-01',
end='1981-05-31',
)
```
![Grand Coulee](docs/_static/grand_coulee_1981_05.png)

Alternatively, certain model outputs deemed most important can be accessed using the BMI interface methods. For example:
Plot a particular output variable (as defined in `config.yaml`) over time:
```python
from mosartwmpy import Model

from mosartwmpy.plotting.plot import plot_reservoir
config_file = 'config.yaml'
mosart_wm = Model()
mosart_wm.initialize()

# get a list of model output variables
mosart_wm.get_output_var_names()
mosart_wm.initialize(config_file)
mosart_wm.update_until()

plot_variable(
model=mosart_wm,
variable='RIVER_DISCHARGE_OVER_LAND_LIQ',
start='1981-05-01',
end='1981-05-31',
log_scale=True,
cmap='winter_r',
)
```
![River network no tiles](docs/_static/river_without_tiles_1981_05.png)

# get the flattened numpy.ndarray of values for an output variable
supply = mosart_wm.get_value_ptr('supply_water_amount')
If `cartopy`, `scipy`, and `geoviews` are installed, tiles can be displayed along with the plot:
```python
plot_variable(
model=mosart_wm,
variable='RIVER_DISCHARGE_OVER_LAND_LIQ',
start='1981-05-01',
end='1981-05-31',
log_scale=True,
cmap='winter_r',
tiles='StamenWatercolor'
)
```
![River network with tiles](docs/_static/river_with_tiles_1981_05.png)

## model coupling

A common use case for `mosartwmpy` is to run coupled with output from the Community Land Model (CLM). To see an example of how to drive `mosartwmpy` with runoff from a coupled model, check out the [Jupyter notebook tutorial](https://github.com/IMMM-SFA/mosartwmpy/blob/main/notebooks/tutorial.ipynb)!

## testing and validation

Expand Down
9 changes: 8 additions & 1 deletion config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -25,4 +25,11 @@ water_management:
read_from_file: true
path: ./input/demand/demand_1981_05.nc
reservoirs:
path: ./input/reservoirs/reservoirs.nc
parameters:
path: ./input/reservoirs/grand_reservoir_parameters.nc
dependencies:
path: ./input/reservoirs/grand_dependency_database.parquet
streamflow:
path: ./input/reservoirs/grand_average_monthly_flow.parquet
demand:
path: ./input/reservoirs/grand_average_monthly_demand.parquet
Binary file added docs/_static/grand_coulee_1981_05.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/river_transport.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/river_with_tiles_1981_05.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/_static/river_without_tiles_1981_05.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 0 additions & 4 deletions launch.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,5 @@
if __name__ == '__main__':

import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

from mosartwmpy import Model

# launch simulation
Expand Down
2 changes: 1 addition & 1 deletion mosartwmpy/_version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.1.0"
__version__ = "0.2.0"
Loading

0 comments on commit 12b5b5c

Please sign in to comment.