Ribasim.config is a submodule of Ribasim to handle the configuration of a Ribasim model. It is implemented using the Configurations package. A full configuration is represented by Config, which is the main API. Ribasim.config is a submodule mainly to avoid name clashes between the configuration sections and the rest of Ribasim.
Store information for a subnetwork used for allocation.
nodeid: All the IDs of the nodes that are in this subnetwork nodeidmapping: Mapping Dictionary; modelnodeid => AGnodeid where such a correspondence exists (all AG node ids are in the values) nodeidmappinginverse: The inverse of nodeidmapping, Dictionary; AG node ID => model node ID Source edge mapping: AG source node ID => subnetwork source edge ID graphallocation: The graph used for the allocation problems capacity: The capacity per edge of the allocation graph, as constrained by nodes that have a maxflowrate problem: The JuMP.jl model for solving the allocation problem Δtallocation: The time interval between consecutive allocation solves
graphflow, graphcontrol: directed graph with vertices equal to ids flow: store the flow on every flow edge edgeidsflow, edgeidscontrol: get the external edge id from (src, dst) edgeconnectiontypeflow, edgeconnectiontypescontrol: get (srcnodetype, dstnodetype) from edge id
if autodiff T = DiffCache{SparseArrays.SparseMatrixCSC{Float64, Int64}, Vector{Float64}} else T = SparseMatrixCSC{Float64, Int} end
nodeid: node ID of the DiscreteControl node; these are not unique but repeated by the amount of conditions of this DiscreteControl node listenfeatureid: the ID of the node/edge being condition on variable: the name of the variable in the condition greaterthan: The threshold value in the condition conditionvalue: The current value of each condition controlstate: Dictionary: node ID => (control state, control state start) logic_mapping: Dictionary: (control node ID, truth state) => control state record: Namedtuple with discrete control information for output
nodeid: node ID of the DiscreteControl node; these are not unique but repeated by the amount of conditions of this DiscreteControl node listenfeatureid: the ID of the node/edge being condition on variable: the name of the variable in the condition greaterthan: The threshold value in the condition conditionvalue: The current value of each condition controlstate: Dictionary: node ID => (control state, control state start) logic_mapping: Dictionary: (control node ID, truth state) => control state record: Namedtuple with discrete control information for results
The Model struct is an initialized model, combined with the Config used to create it and saved outputs. The Basic Model Interface (BMI) is implemented on the Model. A Model can be created from the path to a TOML configuration file, or a Config object.
The Model struct is an initialized model, combined with the Config used to create it and saved results. The Basic Model Interface (BMI) is implemented on the Model. A Model can be created from the path to a TOML configuration file, or a Config object.
nodeid: node ID of the Outlet node active: whether this node is active and thus contributes flow flowrate: target flow rate minflowrate: The minimal flow rate of the outlet maxflowrate: The maximum flow rate of the outlet controlmapping: dictionary from (nodeid, controlstate) to target flow rate ispid_controlled: whether the flow rate of this outlet is governed by PID control
PID control currently only supports regulating basin levels.
nodeid: node ID of the PidControl node active: whether this node is active and thus sets flow rates listennodeid: the id of the basin being controlled pidparams: a vector interpolation for parameters changing over time. The parameters are respectively target, proportional, integral, derivative, where the last three are the coefficients for the PID equation. error: the current error; basintarget - currentlevel
nodeid: node ID of the Pump node active: whether this node is active and thus contributes flow flowrate: target flow rate minflowrate: The minimal flow rate of the pump maxflowrate: The maximum flow rate of the pump controlmapping: dictionary from (nodeid, controlstate) to target flow rate ispid_controlled: whether the flow rate of this pump is governed by PID control
Rating curve from level to discharge. The rating curve is a lookup table with linear interpolation in between. Relation can be updated in time, which is done by moving data from the time field into the tables, which is done in the update_tabulated_rating_curve callback.
Type parameter C indicates the content backing the StructVector, which can be a NamedTuple of Vectors or Arrow Primitives, and is added to avoid type instabilities.
nodeid: node ID of the TabulatedRatingCurve node active: whether this node is active and thus contributes flows tables: The current Q(h) relationships time: The time table used for updating the tables controlmapping: dictionary from (nodeid, controlstate) to Q(h) and/or active state
demand: water flux demand of user per priority over time active: whether this node is active and thus demands water allocated: water flux currently allocated to user per priority returnfactor: the factor in [0,1] of how much of the abstracted water is given back to the system minlevel: The level of the source basin below which the user does not abstract priorities: All used priority values. Each user has a demand for all these priorities, which is always 0.0 if it is not provided explicitly.
Parse a TOML file to a Config. Keys can be overruled using keyword arguments. To overrule keys from a subsection, e.g. dt from the solver section, use underscores: solver_dt.
Add the basin allocation constraints to the allocation problem; the allocations to the basins are bounded from above by the basin demand (these are set before each allocation solve). The constraint indices are allocation graph basin node IDs.
Add the flow capacity constraints to the allocation problem. Only finite capacities get a constraint. The constraint indices are the allocation graph edge IDs.
Add the source constraints to the allocation problem. The actual threshold values will be set before each allocation solve. The constraint indices are the allocation graph source node IDs.
Constraint: flow over source edge <= source flow in subnetwork
Add the objective function to the allocation problem. Objective function: linear combination of allocations to the basins and users, where basin allocations get a weight of 1.0 and user allocations get a weight of 2^(-priority index).
Add the basin allocation variables A_basin to the allocation problem. The variable indices are the allocation graph basin node IDs. Non-negativivity constraints are also immediately added to the basin allocation variables.
Add the user allocation variables Auser{i} to the allocation problem. The variable name indices i are the allocation graph user node IDs, The variable indices are the priorities.
Add the flow variables F to the allocation problem. The variable indices are the allocation graph edge IDs. Non-negativivity constraints are also immediately added to the flow variables.
Update the allocation optimization problem for the given subnetwork with the problem state and flows, solve the allocation problem and assign the results to the users.
Remove user return flow edges that are upstream of the user itself, and collect the IDs of the allocation graph node IDs of the users that do not have this problem.
Create the different callbacks that are used to store output and feed the simulation with new data. The different callbacks are combined to a CallbackSet that goes to the integrator. Returns the CallbackSet and the SavedValues for flow.
Create the different callbacks that are used to store results and feed the simulation with new data. The different callbacks are combined to a CallbackSet that goes to the integrator. Returns the CallbackSet and the SavedValues for flow.
Convert a Real that represents the seconds passed since the simulation start to the nearest DateTime. This is used to convert between the solver’s inner float time, and the calendar.
For an element id and a vector of elements ids, get the range of indices of the last consecutive block of id. Returns the empty range 1:0 if id is not in ids.
Find the index of element x in a sorted collection a. Returns the index of x if it exists, or nothing if it doesn’t. If x occurs more than once, throw an error.
Get a sparse matrix whose sparsity matches the sparsity of the Jacobian of the ODE problem. All nodes are taken into consideration, also the ones that are inactive.
In Ribasim the Jacobian is typically sparse because each state only depends on a small number of other states.
Note: the name ‘prototype’ does not mean this code is a prototype, it comes from the naming convention of this sparsity structure in the differentialequations.jl docs.
Get the current water level of a node ID. The ID can belong to either a Basin or a LevelBoundary. storage: tells ForwardDiff whether this call is for differentiation or not
Load data from Arrow files if available, otherwise the GeoPackage. Always returns a StructVector of the given struct type T, which is empty if the table is not found. This function validates the schema, and enforces the required sort order.
Load data from Arrow files if available, otherwise the database. Always returns a StructVector of the given struct type T, which is empty if the table is not found. This function validates the schema, and enforces the required sort order.
Process the data in the static and time tables for a given node type. The ‘defaults’ named tuple dictates how missing data is filled in. ‘time_interpolatables’ is a vector of Symbols of parameter names for which a time interpolation (linear) object must be constructed. The control mapping for DiscreteControl is also constructed in this function. This function currently does not support node states that are defined by more than one row in a table, as is the case for TabulatedRatingCurve.
Run a Model, given a path to a TOML configuration file, or a Config object. Running a model includes initialization, solving to the end with [solve!](@ref) and writing output with BMI.finalize.
Run a Model, given a path to a TOML configuration file, or a Config object. Running a model includes initialization, solving to the end with [solve!](@ref) and writing results with BMI.finalize.
Convert a DateTime to a float that is the number of seconds since the start of the simulation. This is used to convert between the solver’s inner float time, and the calendar.
From a timeseries table time, load the most recent applicable data into table. table must be a NamedTuple of vectors with all variables that must be loaded. The most recent applicable data is non-NaN data for a given ID that is on or before t.
Update table at row index i, with the values of a given row. table must be a NamedTuple of vectors with all variables that must be loaded. The row must contain all the column names that are present in the table. If a value is NaN, it is not set.
The controlled basin affects itself and the basins upstream and downstream of the controlled pump affect eachother if there is a basin upstream of the pump. The state for the integral term and the controlled basin affect eachother, and the same for the integral state and the basin upstream of the pump if it is indeed a basin.
If both the unique node upstream and the unique node downstream of these nodes are basins, then these directly depend on eachother and affect the Jacobian 2x Basins always depend on themselves.
If both the unique node upstream and the nodes down stream (or one node further if a fractional flow is in between) are basins, then the downstream basin depends on the upstream basin(s) and affect the Jacobian as many times as there are downstream basins Upstream basins always depend on themselves.
Check that nodes that have fractional flow outneighbors do not have any other type of outneighbor, that the fractions leaving a node add up to ≈1 and that the fractions are non-negative.
Ribasim is typically used as a command-line interface (CLI). It is distributed as a .zip archive, that must be downloaded and unpacked. It can be placed anywhere, however it is important that the contents of the zip file are kept together in a directory. The Ribasim CLI executable is in the bin directory.
To check whether the installation was performed successfully, run ribasim with no arguments in the command line. This will give the following message:
-
Usage: ribasim 'path/to/config.toml'
+
Usage: ribasim 'path/to/model/ribasim.toml'
1 Configuration file
Ribasim has a single configuration file, which is written in the TOML format. It contains settings, as well as paths to other input and output files.
@@ -323,19 +323,19 @@
1 Configuration f
update_timestep=86400.0# optional, default 1 day# input files
-geopackage="model.gpkg"# required
+database="database.gpkg"# required
-[output]
-# These output files are always written
-flow="output/flow.arrow"# optional, default "output/flow.arrow"
-basin="output/basin.arrow"# optional, default "output/basin.arrow"
-control="output/control.arrow"# optional, default "output/control.arrow"
+[results]
+# These results files are always written
+flow="results/flow.arrow"# optional, default "results/flow.arrow"
+basin="results/basin.arrow"# optional, default "results/basin.arrow"
+control="results/control.arrow"# optional, default "results/control.arrow"compression="zstd"# optional, default "zstd", also supports "lz4"compression_level=6# optional, default 6
-# Specific tables can also go into Arrow files rather than the GeoPackage.
+# Specific tables can also go into Arrow files rather than the database.# For large tables this can benefit from better compressed file sizes.
-# This is optional, tables are retrieved from the GeoPackage if not specified in the TOML.
+# This is optional, tables are retrieved from the database if not specified in the TOML.[basin]time="basin/time.arrow"
@@ -359,21 +359,21 @@
1 Configuration f
timing=false# optional, whether to log debug timing statements
1.1 Solver settings
-
The solver section in the configuration file is entirely optional, since we aim to use defaults that will generally work well. Common reasons to modify the solver settings are to adjust the calculation or output stepsizes: adaptive, dt, and saveat. If your model does not converge, or your performance is lower than expected, it can help to adjust other solver settings as well.
+
The solver section in the configuration file is entirely optional, since we aim to use defaults that will generally work well. Common reasons to modify the solver settings are to adjust the calculation or result stepsizes: adaptive, dt, and saveat. If your model does not converge, or your performance is lower than expected, it can help to adjust other solver settings as well.
The default solver algorithm = "QNDF", which is a multistep method similar to Matlab’s ode15s(Shampine and Reichelt 1997). It is an implicit method that supports the default adaptive = true timestepping. The full list of available solvers is: QNDF, Rosenbrock23, TRBDF2, Rodas5, KenCarp4, Tsit5, RK4, ImplicitEuler, Euler. Information on the solver algorithms can be found on the ODE solvers page.
The dt controls the stepsize. When adaptive = true, dt only applies to the initial stepsize, and by default it is automatically determined. When adaptive = false a suitable dt must always be provided. The value is in seconds, so dt = 3600.0 corresponds to an hourly timestep. When adaptive = true, dtmin and dtmax control the minimum and maximum allowed dt. By default these depend on the problem and algorithm. If a smaller dt than dtmin is needed to meet the set error tolerances, the simulation stops, unless force_dtmin is set to true. force_dtmin is off by default to ensure an accurate solution.
-
By default the calculation and output stepsize are the same, with saveat = [], which will save every timestep. saveat can be a number, which is the saving interval in seconds, or it can be a list of numbers, which are the times in seconds since start that are saved. For instance, saveat = 86400.0 will save output after every day that passed.
+
By default the calculation and result stepsize are the same, with saveat = [], which will save every timestep. saveat can be a number, which is the saving interval in seconds, or it can be a list of numbers, which are the times in seconds since start that are saved. For instance, saveat = 86400.0 will save results after every day that passed.
The Jacobian matrix provides information about the local sensitivity of the model with respect to changes in the states. For implicit solvers it must be calculated often, which can be expensive to do. There are several methods to do this. By default Ribasim uses a Jacobian derived automatically using ForwardDiff.jl with memory management provided by PreallocationTools.jl. If this is not used by setting autodiff = false, the Jacobian is calculated with a finite difference method, which can be less accurate and more expensive.
By default the Jacobian matrix is a sparse matrix (sparse = true). Since each state typically only depends on a small number of other states, this is generally more efficient, especially for larger models. The sparsity structure is calculated from the network and provided as a Jacobian prototype to the solver. For small or highly connected models it could be faster to use a dense Jacobian matrix instead by setting sparse = false.
The total maximum number of iterations maxiters = 1e9, can normally stay as-is unless doing extremely long simulations.
The absolute and relative tolerance for adaptive timestepping can be set with abstol and reltol. For more information on these and other solver options, see the DifferentialEquations.jl docs.
-
-
2 GeoPackage and Arrow tables
-
The input and output tables described below all share that they are tabular files. The Node and Edge tables always have to be in the GeoPackage file, and output is always written to Apache Arrow files, sometimes also known as Feather files. All other tables can either be in the GeoPackage or in separate Arrow files that are listed in the TOML as described above.
+
+
2 GeoPackage database and Arrow tables
+
The input and output tables described below all share that they are tabular files. The Node and Edge tables always have to be in the GeoPackage database file, and results are always written to Apache Arrow files, sometimes also known as Feather files. All other tables can either be in the database or in separate Arrow files that are listed in the TOML as described above.
For visualization, the Node and Edge tables typically have associated geometries. GeoPackage was used since it provides a standardized way to store tables with (and without) geometry columns in a SQLite database. If, like Ribasim, you can ignore the geometries, a GeoPackage is easy to read using SQLite libraries, which are commonly available. Furthermore GeoPackage can be updated in place when working on a model.
-
Arrow was chosen since it is standardized, fast, simple and flexible. It can be read and written by many different software packages. In Ribasim we use Arrow.jl. Output is written to Arrow, since for long runs output can producs tables with many rows. Arrow is well suited for large tabular datasets, and file size is kept small by using compression. The Arrow input files can be compressed with LZ4 or Zstd compression. Furthermore, in some of the columns, a small amount of different values are repeated many times. To reduce file sizes it may be a good idea to apply dictionary encoding to those columns.
+
Arrow was chosen since it is standardized, fast, simple and flexible. It can be read and written by many different software packages. In Ribasim we use Arrow.jl. Results are written to Arrow, since for long runs Ribasim can produce tables with many rows. Arrow is well suited for large tabular datasets, and file size is kept small by using compression. The Arrow input files can be compressed with LZ4 or Zstd compression. Furthermore, in some of the columns, a small amount of different values are repeated many times. To reduce file sizes it may be a good idea to apply dictionary encoding to those columns.
Below we give details per file, in which we describe the schema of the table using a syntax like this:
@@ -432,7 +432,7 @@
3 Node
-
The available node types as of this writing are listed as the top level bullets below. The sub-bullets indicate which tables are associated to the node type. The table name is the name it must have in the GeoPackage if it is stored there.
+
The available node types as of this writing are listed as the top level bullets below. The sub-bullets indicate which tables are associated to the node type. The table name is the name it must have in the database if it is stored there.
Basin: stores water
@@ -701,9 +701,9 @@
We use the symbol \(A\) for area, \(h\) for level and \(S\) for storage. The profile provides a function \(A(h)\) for each basin. Internally this get converted to two functions, \(A(S)\) and \(h(S)\), by integrating over the function, setting the storage to zero for the bottom of the profile. The minimum area cannot be zero to avoid numerical issues. The maximum area is used to convert the precipitation flux into an inflow.
-
-
5.4 Basin output
-
The basin table contains outputs of the storage and level of each basin at every solver timestep. The initial condition is also written to the file.
+
+
5.4 Basin results
+
The basin table contains results of the storage and level of each basin at every solver timestep. The initial condition is also written to the file.
@@ -737,9 +737,9 @@
The table is sorted by time, and per time it is sorted by node_id.
-
-
5.5 Flow output
-
The flow table contains outputs of the flow on every edge in the model, for each solver timestep.
+
+
5.5 Flow results
+
The flow table contains results of the flow on every edge in the model, for each solver timestep.
@@ -1526,8 +1526,8 @@
<
-
-
16.3 DiscreteControl output
+
+
16.3 DiscreteControl results
The control table contains a record of each change of control state: when it happened, which control node was involved, to which control state it changed and based on which truth state.
Write the contents of the model to a GeoPackage and a TOML configuration file.
+
Write the contents of the model to a database and a TOML configuration file.
@@ -478,8 +472,8 @@
1.2.9 write
Model.write(directory)
-
Write the contents of the model to a GeoPackage and a TOML configuration file.
-
If directory does not exist, it is created before writing. The GeoPackage and TOML file will be called {modelname}.gpkg and {modelname}.toml respectively.
+
Write the contents of the model to a database and a TOML configuration file.
+
If directory does not exist, it is created before writing.
Open example model basic.gpkg or create a new model.
+
Open example model database.gpkg or create a new model.
-
If you are working with an unknown CRS, right click the model GeoPackage group in Layers, and click “Set Group CRS…”.
+
If you are working with an unknown CRS, right click the model database group in Layers, and click “Set Group CRS…”.
Unzip the Ribasim command line interface, ribasim_cli.zip
-
Open your terminal and go to the directory where your TOML is stored. Now run path/to/ribasim_cli/ribasim basic.toml. Adjust the path to the ribasim_cli folder to where you placed it. This runs the model.
-
In your model directory there is now an output/ folder with basin.arrow and flow.arrow output files.
+
Open your terminal and go to the directory where your TOML is stored. Now run path/to/ribasim_cli/ribasim ribasim.toml. Adjust the path to the ribasim_cli folder to where you placed it. This runs the model.
+
In your model directory there is now a results/ folder with basin.arrow and flow.arrow output files.
1.5 Inspect results
-
-
1.5.1 Associate output to iMOD time series window
+
+
1.5.1 Associate results to iMOD time series window
In QGIS select the model group.
-
In the Ribasim plugin widget, select the Output tab and click “Associate Output”.
+
In the Ribasim plugin widget, select the Results tab and click “Associate Results”.