Merge pull request #30 from Deltares/geodataset

Geodataset

docs/api/api_index.rst

@@ -130,14 +130,14 @@ SFINCS workflows
 .. autosummary::
    :toctree: ../generated/
 
-   workflows.parse_region
-   workflows.get_basin_geometry
    workflows.mask_topobathy
    workflows.merge_topobathy
    workflows.cn_to_s
    workflows.landuse
    workflows.snap_discharge
-   workflows.resample_time
+   workflows.river_inflow_points
+   workflows.river_outflow_points
+
 
 .. _methods:
 

docs/changelog.rst

@@ -8,12 +8,17 @@ The format is based on `Keep a Changelog`_, and this project adheres to
 [Unreleased]
 ------------
 
+v0.2.0 (2 August 2021)
+---------------------
+
 Bugfix
 ^^^^^^
 - scsfile variable changed to maximum soil moisture retention [inch]; was curve number [-]
+- fix setting delimited text based geodatasets for h and q forcing.
 
 Changed
 ^^^^^^^
+- Bumped minimal hydromt vesion to 0.4.2
 - splitted ``setup_basemaps`` into multiple smaller methods: ``setup_merge_topobathy``, ``setup_mask`` and ``setup_bounds``
 - separated many low-level methods into utils.py and plots.py
 - save bzs/bzd & dis/src only as GeoDataArray at forcing and do not copy the locations at staticgeoms.
@@ -23,7 +28,7 @@ Changed
 
 Added
 ^^^^^
-support for:
+support for SFINCS files:
 
 - structures: sfincs.thd & sfincs.weir
 - results: sfincs_map.nc & sfincs_his.nc
@@ -40,6 +45,12 @@ new methods:
 - ``read_results`` 
 - ``update_spatial_attrs`` and ``get_spatial_attrs`` (previously part of read_staticmaps)
 
+new workflows: 
+
+- ``merge_topobathy``
+- ``mask_topobathy``
+- ``snap_discharge``
+- ``river_inflow_points`` & ``river_outflow_points`` 
 
 Documentation
 ^^^^^^^^^^^^^
@@ -50,7 +61,7 @@ Deprecated
 ^^^^^^^^^^^
 - ``setup_p_gridded``
 
-v0.0.1 (18 May 2021)
+v0.1.0 (18 May 2021)
 --------------------
 Noticeable changes are a new ``setup_river_inflow`` and ``setup_river_outflow`` methods
 

docs/user_guide/sfincs.rst

@@ -1,4 +1,3 @@
-.. _model_components:
 .. currentmodule:: hydromt_sfincs.sfincs
 
 =====================
@@ -27,6 +26,8 @@ Note that the order in which the components are listed in the ini file is import
 
 For python users all SFINCS attributes and methods are available, see :ref:`api_model`
 
+.. _model_components:
+
 SfincsModel setup components
 ============================
 
@@ -35,7 +36,9 @@ is provided in the table below. When using hydromt from the command line only th
 setup components are exposed. Click on header to get a full overview or directly on
 a specific method see its documenation.  
 
-.. list-table:: SfincsModel setup components
+.. _general_table:
+
+.. list-table:: General setup components
    :widths: 20 25 25 30
    :header-rows: 1
 
@@ -44,62 +47,75 @@ a specific method see its documenation.
      - :ref:`workflows <workflows>`
      - :ref:`low-level methods <methods>`
    * - sfincs.inp
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_config`
-     - :py:meth:`~hydromt_sfincs.workflows.parse_region`:sup:`1` :py:meth:`~hydromt_sfincs.workflows.get_basin_geometry`:sup:`1`
-     - :py:meth:`~hydromt_sfincs.read_inp` :py:meth:`~hydromt_sfincs.write_inp` :py:meth:`~hydromt_sfincs.get_spatial_attrs`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_config`
+     - :py:func:`~hydromt.workflows.parse_region`:sup:`1` :py:func:`~hydromt.workflows.get_basin_geometry`:sup:`1`
+     - :py:func:`~hydromt_sfincs.read_inp` :py:func:`~hydromt_sfincs.write_inp` :py:func:`~hydromt_sfincs.get_spatial_attrs`
    * - depfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_basemaps` :py:meth:`~hydromt_sfincs.SfincsModel.setup_merge_topobathy`
-     - :py:meth:`~hydromt_sfincs.workflows.merge_topobathy`
-     - :py:meth:`~hydromt_sfincs.read_binary_map` :py:meth:`~hydromt_sfincs.write_binary_map`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_basemaps` :py:func:`~hydromt_sfincs.SfincsModel.setup_merge_topobathy`
+     - :py:func:`~hydromt_sfincs.workflows.merge_topobathy`
+     - :py:func:`~hydromt_sfincs.read_binary_map` :py:func:`~hydromt_sfincs.write_binary_map`
    * - mskfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_mask` :py:meth:`~hydromt_sfincs.SfincsModel.setup_bounds` :py:meth:`~hydromt_sfincs.SfincsModel.setup_river_outflow`
-     - :py:meth:`~hydromt_sfincs.workflows.mask_topobathy`
-     - :py:meth:`~hydromt_sfincs.read_binary_map` :py:meth:`~hydromt_sfincs.write_binary_map` :py:meth:`~hydromt_sfincs.utils.mask_bounds`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_mask` :py:func:`~hydromt_sfincs.SfincsModel.setup_bounds` :py:func:`~hydromt_sfincs.SfincsModel.setup_river_outflow`
+     - :py:func:`~hydromt_sfincs.workflows.mask_topobathy`
+     - :py:func:`~hydromt_sfincs.read_binary_map` :py:func:`~hydromt_sfincs.write_binary_map` :py:func:`~hydromt_sfincs.utils.mask_bounds`
    * - indfile
      - 
      - 
-     - :py:meth:`~hydromt_sfincs.read_binary_map_index` :py:meth:`~hydromt_sfincs.write_binary_map_index`
+     - :py:func:`~hydromt_sfincs.read_binary_map_index` :py:func:`~hydromt_sfincs.write_binary_map_index`
    * - manningfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_manning_roughness`
-     - :py:meth:`~hydromt_sfincs.workflows.landuse`
-     - :py:meth:`~hydromt_sfincs.read_binary_map` :py:meth:`~hydromt_sfincs.write_binary_map`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_manning_roughness`
+     - :py:func:`~hydromt_sfincs.workflows.landuse`
+     - :py:func:`~hydromt_sfincs.read_binary_map` :py:func:`~hydromt_sfincs.write_binary_map`
    * - scsfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_cn_infiltration`
-     - :py:meth:`~hydromt_sfincs.workflows.cn_to_s`
-     - :py:meth:`~hydromt_sfincs.read_binary_map` :py:meth:`~hydromt_sfincs.write_binary_map`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_cn_infiltration`
+     - :py:func:`~hydromt_sfincs.workflows.cn_to_s`
+     - :py:func:`~hydromt_sfincs.read_binary_map` :py:func:`~hydromt_sfincs.write_binary_map`
    * - obsfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_gauges`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_gauges`
      -
-     - :py:meth:`~hydromt_sfincs.read_xy` :py:meth:`~hydromt_sfincs.write_xy`
+     - :py:func:`~hydromt_sfincs.read_xy` :py:func:`~hydromt_sfincs.write_xy`
    * - thd- & weirfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_structures`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_structures`
      -
-     - :py:meth:`~hydromt_sfincs.read_structures` :py:meth:`~hydromt_sfincs.write_structures` :py:meth:`~hydromt_sfincs.utils.gdf2structures` :py:meth:`~hydromt_sfincs.utils.structures2gdf`
+     - :py:func:`~hydromt_sfincs.read_structures` :py:func:`~hydromt_sfincs.write_structures` :py:func:`~hydromt_sfincs.utils.gdf2structures` :py:func:`~hydromt_sfincs.utils.structures2gdf`
+
+.. _focing_table:
+
+.. list-table:: Forcing setup components
+   :widths: 20 25 25 30
+   :header-rows: 1
+
+   * - SFINCS file
+     - :ref:`setup components <components>`
+     - :ref:`workflows <workflows>`
+     - :ref:`low-level methods <methods>`
    * - bnd- & bzsfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_h_forcing`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_h_forcing`
      -
-     - :py:meth:`~hydromt_sfincs.read_timeseries` :py:meth:`~hydromt_sfincs.write_timeseries` :py:meth:`~hydromt_sfincs.read_xy` :py:meth:`~hydromt_sfincs.write_xy`
+     - :py:func:`~hydromt_sfincs.read_timeseries` :py:func:`~hydromt_sfincs.write_timeseries` :py:func:`~hydromt_sfincs.read_xy` :py:func:`~hydromt_sfincs.write_xy`
    * - src- & disfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_river_inflow` :py:meth:`~hydromt_sfincs.SfincsModel.setup_q_forcing` :py:meth:`~hydromt_sfincs.SfincsModel.setup_q_forcing_from_grid`
-     - :py:meth:`~hydromt_sfincs.workflows.snap_discharge`
-     - :py:meth:`~hydromt_sfincs.read_timeseries` :py:meth:`~hydromt_sfincs.write_timeseries` :py:meth:`~hydromt_sfincs.read_xy` :py:meth:`~hydromt_sfincs.write_xy`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_river_inflow` :py:func:`~hydromt_sfincs.SfincsModel.setup_q_forcing` :py:func:`~hydromt_sfincs.SfincsModel.setup_q_forcing_from_grid`
+     - :py:func:`~hydromt_sfincs.workflows.snap_discharge`
+     - :py:func:`~hydromt_sfincs.read_timeseries` :py:func:`~hydromt_sfincs.write_timeseries` :py:func:`~hydromt_sfincs.read_xy` :py:func:`~hydromt_sfincs.write_xy`
    * - precipfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_p_forcing` :py:meth:`~hydromt_sfincs.SfincsModel.setup_p_forcing_from_grid`
-     - :py:meth:`~hydromt_sfincs.workflows.resample_time`:sup:`1`
-     - :py:meth:`~hydromt_sfincs.read_timeseries` :py:meth:`~hydromt_sfincs.write_timeseries`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_p_forcing` :py:func:`~hydromt_sfincs.SfincsModel.setup_p_forcing_from_grid`
+     - :py:func:`~hydromt.workflows.resample_time`:sup:`1`
+     - :py:func:`~hydromt_sfincs.read_timeseries` :py:func:`~hydromt_sfincs.write_timeseries`
    * - netamprfile
-     - :py:meth:`~hydromt_sfincs.SfincsModel.setup_p_forcing_from_grid`
-     - :py:meth:`~hydromt_sfincs.workflows.resample_time`:sup:`1`
+     - :py:func:`~hydromt_sfincs.SfincsModel.setup_p_forcing_from_grid`
+     - :py:func:`~hydromt.workflows.resample_time`:sup:`1`
      -
 
 :sup:`1`) Imported from hydromt core package
 
 SFINCS datamodel
 ================
 
-The following table provides an overview of which ``SfincsModel`` attribute contains 
-which SFINCS in- and output files. The files are read and written with the associated 
-read- and write- methods, i.e. ``read_config`` and ``write_config`` for the ``config`` attribute. 
+The following table provides an overview of which :py:class:`~hydromt_sfincs.SfincsModel` 
+attribute contains which SFINCS in- and output files. The files are read and written with the associated 
+read- and write- methods, i.e. :py:func:`~hydomt_sfincs.sfincs.SfincsModel.read_config` 
+and :py:func:`~hydomt_sfincs.sfincs.SfincsModel.write_config` for the 
+:py:attr:`~hydomt_sfincs.sfincs.SfincsModel.config`  attribute. 
 
 Note that the indfile is not part of the staticmaps dataset but created based on 
 the mskfile upon writing and used for reading staticmaps.
@@ -109,7 +125,7 @@ the mskfile upon writing and used for reading staticmaps.
    :widths: 30 70
    :header-rows: 1
 
-   * - ``SfincsModel`` attribute
+   * - :py:class:`~hydromt_sfincs.SfincsModel` attribute
      - SFINCS files
    * - :py:attr:`~hydromt_sfincs.SfincsModel.config`
      - sfincs.inp

examples/build_coastal_model.ipynb

@@ -254,10 +254,10 @@
     "* `[setup_merge_topobathy]`: add additional datasource for bathymetry data to merge with the basemap elevation data.\n",
     "* `[setup_mask]`: set valid model cells based on the topobathy data and a minimal depth boundary (more options available).\n",
     "\n",
-    "* `[setup_cn_infiltration]`: infiltration curve number maps, see [sfincs docs](https://sfincs.readthedocs.io/en/latest/input.html?highlight=curve#spatially-varying-curve-number) for more information\n",
+    "* `[setup_cn_infiltration]`: potential maximum soil retention maps to be used in the curve number method, see [sfincs docs](https://sfincs.readthedocs.io/en/latest/input.html?highlight=curve#spatially-varying-curve-number) for more information\n",
     "* `[setup_manning_roughness]`: configuration of the Manning roughness values, typically consisting of a land use or land cover map, and a lookup table that relates land use classes to manning roughness values in sec m$^{-1/3}$\n",
     "* `[setup_bounds]`: set cells at the model domain edge and within the sea polygon to waterlevel boundary cells.\n",
-    "* `[setup_h_forcing]`: water level forcing data. This can be provided as a NetCDF file following CF-conventions for point time series data.\n",
+    "* `[setup_h_forcing]`: water level forcing data. This can be provided from a set of csv files, see data_sources.yml, but also netcdf files, see [hydromt docs](https://deltares.github.io/hydromt/latest/user_guide/data.html). \n",
     "* `[setup_p_forcing_from_grid]`: gridded precipitation data, that can be used to do compound flood simulations. The `[setup_cn_infiltration]` is then quite essential to include."
    ]
   },
@@ -276,7 +276,7 @@
    },
    "outputs": [],
    "source": [
-    "!hydromt build sfincs ./venice_sfincs_150_full \"{'bbox': [12.05,45.30,12.85,45.65]}\" -r 150 -i sfincs_coastal.ini -vv"
+    "!hydromt build sfincs ./venice_sfincs_150_full \"{'bbox': [12.05,45.30,12.85,45.65]}\" -r 150 -i sfincs_coastal.ini -d data_catalog.yml -vv"
    ]
   },
   {
@@ -341,7 +341,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -355,7 +355,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.4"
+   "version": "3.9.6"
   }
  },
  "nbformat": 4,

examples/build_from_py.ipynb

@@ -102,7 +102,7 @@
     "# export a model to gis files using hydromt (only for sake of the example)\n",
     "mod0 = SfincsModel(root='sfincs_riverine', mode='r')\n",
     "mod0.read()\n",
-    "mod0.write_raster(variables=['staticmaps.dep'], root='sfincs_gis')"
+    "mod0.write_raster(variables=['staticmaps.dep'], root='data')"
    ]
   },
   {
@@ -144,7 +144,7 @@
    "source": [
     "# read all data to a Dataset\n",
     "# note that the names are taken from the file basenames\n",
-    "da_dep = hydromt.open_raster('sfincs_gis/dep.tif')\n",
+    "da_dep = hydromt.open_raster('data/dep.tif')\n",
     "da_dep"
    ]
   },
@@ -171,7 +171,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "mod = SfincsModel(root='sfincs_from_gis', mode='w')\n",
+    "mod = SfincsModel(root='sfincs_from_py', mode='w')\n",
     "# Note this is still an empty model with no maps\n",
     "len(mod.staticmaps) == 0"
    ]
@@ -181,9 +181,9 @@
    "id": "0cf2009d",
    "metadata": {},
    "source": [
-    "We can add staticmaps to the model using the `.set_staticmaps` method. Here it is important that the model layers are on a identical grid, otherwise these will not be excepted. \n",
+    "We can add staticmaps to the model using the `set_staticmaps()` method. Here it is important that the model layers are on a identical grid, otherwise these will not be excepted. \n",
     "\n",
-    "Note that geotiff raster typically has a N->S orientation while the SFINCS model uses a S->N orientation. If the staticmaps are  kept in N->S orientation the y-axis will be flipped by the `.set_staticmaps` method. Compare the y dimension between ds and .staticmaps to check."
+    "Note that geotiff raster typically has a N->S orientation while the SFINCS model uses a S->N orientation. If the staticmaps are  kept in N->S orientation the y-axis will be flipped by the `set_staticmaps()` method. Compare the y dimension between ds and .staticmaps to check."
    ]
   },
   {
@@ -217,6 +217,26 @@
     "mod.config  # inspect config"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "e8695819",
+   "metadata": {},
+   "source": [
+    "Next we need to define the mask and boundaries where water can leave the model domain. For a riverine application this can be done using the ``setup_river_outflow()`` method to find set cells boundary cells within a range from river outflow points to an open boundary. The more general ``setup_boundary()`` method can be used to setup both open and water level boundary cells."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "f0a01245",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# setup mask with open boundaries at the downstream end of \n",
+    "mod.setup_mask()  # sets mask based on valid dep cells\n",
+    "mod.setup_river_outflow()  # use default settings"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -243,7 +263,7 @@
    "source": [
     "The SFINCS discharge forcing is set based on a combination of a the src (xy point locations) and dis (timeseries) files. In hydroMT the src locations are represented by a geopandas.GeoDataFrame in the staticgeoms attribute and dis timeseries by a xarray.DataArray with 'time' and 'index' dimensions in the forcing attribute. SFINCS waterlevel forcing can be set similarly using 'bzd' (xy point locations) and 'bzs' (timeseries) and precipitation with 'precip' (timeseries). \n",
     "\n",
-    "NOTE: If csv or netcdf files with location and timeseries are available, the `setup_q_forcing` method is recommended!"
+    "NOTE: If csv or netcdf files with location and timeseries are available, the `setup_q_forcing()` method is recommended. River inflow points can also be found based on a flow direction dataset using `setup_river_inflow()`"
    ]
   },
   {
@@ -373,7 +393,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -387,7 +407,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.4"
+   "version": "3.9.6"
   }
  },
  "nbformat": 4,

examples/build_riverine_model.ipynb

@@ -293,7 +293,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3",
+   "display_name": "Python 3 (ipykernel)",
    "language": "python",
    "name": "python3"
   },
@@ -307,7 +307,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.4"
+   "version": "3.9.6"
   }
  },
  "nbformat": 4,

examples/data/gtsm_locations.csv

@@ -0,0 +1,7 @@
+index,x,y
+1,291821.42,5027884.99
+2,298859.33,5032543.89
+3,311710.14,5040289.96
+4,318671.89,5043342.74
+5,325626.55,5046403.05
+6,331431.92,5049500.47