Refactor docs to follow diataxis concept

docs/_quarto.yml

@@ -9,62 +9,84 @@ website:
   navbar:
     logo: https://user-images.githubusercontent.com/4471859/224825908-bee7e044-bc6b-4561-8b08-5d330cce3ed5.png
     left:
-      - text: "Julia core"
-        file: core/index.qmd
-      - text: "Python tooling"
-        file: python/index.qmd
-      - text: "QGIS plugin"
-        file: qgis/index.qmd
-      - text: "Contributing"
-        file: contribute/index.qmd
+      - text: "About"
+        file: index.qmd
+      - text: "Get Started"
+        file: tutorials/index.qmd
+      - text: "Guides"
+        file: guides/index.qmd
+      - text: "Background"
+        file: background/concept.qmd
+      - text: "Reference"
+        file: reference/index.qmd
+      - text: "Changelog"
+        file: changelog.md
     right:
       - icon: github
         href: https://github.com/Deltares/Ribasim
         aria-label: GitHub
 
   sidebar:
-    - title: "Julia core"
+    - title: "Get Started"
       contents:
-        - core/index.qmd
-        - core/modelconcept.qmd
-        - core/usage.qmd
-        - core/validation.qmd
-        - core/equations.qmd
-        - core/allocation.qmd
-        - core/numerics.qmd
-        - core/coupling.qmd
-        - section: "Nodes"
+        - tutorials/index.qmd
+        - tutorials/julia.qmd
+        - tutorials/python.qmd
+        - tutorials/qgis.qmd
+
+    - title: "Guides"
+      contents:
+        - guides/index.qmd
+        - section: "For end-users"
           contents:
-            - core/node/basin.qmd
-            - core/node/tabulated-rating-curve.qmd
-            - core/node/pump.qmd
-            - core/node/outlet.qmd
-            - core/node/user-demand.qmd
-            - core/node/flow-demand.qmd
-            - core/node/level-demand.qmd
-            - core/node/flow-boundary.qmd
-            - core/node/level-boundary.qmd
-            - core/node/linear-resistance.qmd
-            - core/node/manning-resistance.qmd
-            - core/node/fractional-flow.qmd
-            - core/node/terminal.qmd
-            - core/node/discrete-control.qmd
-            - core/node/pid-control.qmd
-    - title: "Python tooling"
+          - guides/examples.ipynb
+          - guides/python.qmd
+          - guides/coupling.qmd
+        - section: "For developers"
+          contents:
+          - guides/core.qmd
+          - guides/qgis.qmd
+          - dev/python.qmd
+          - guides/addnode.qmd
+          - guides/release.qmd
+
+    - title: "Background"
       contents:
-        - python/index.qmd
-        - python/examples.ipynb
-        - python/reference/index.qmd
-        - python/test-models.qmd
-    - title: "Contributing"
+        - background/concept.qmd
+        - background/usage.qmd
+        - section: "Numerics"
+          contents:
+          - background/modelconcept.qmd
+          - background/equations.qmd
+          - background/numerics.qmd
+        - section: "Implementation"
+          contents:
+          - background/core.qmd
+          - background/validation.qmd
+          - background/allocation.qmd
+          - background/ci.qmd
+
+    - title: "Reference"
       contents:
-        - contribute/index.qmd
-        - contribute/core.qmd
-        - contribute/python.qmd
-        - contribute/qgis.qmd
-        - contribute/addnode.qmd
-        - contribute/ci.qmd
-        - contribute/release.qmd
+        - reference/index.qmd
+        - reference/test-models.qmd
+        - section: "Nodes"
+          contents:
+            - reference/node/basin.qmd
+            - reference/node/tabulated-rating-curve.qmd
+            - reference/node/pump.qmd
+            - reference/node/outlet.qmd
+            - reference/node/user-demand.qmd
+            - reference/node/flow-demand.qmd
+            - reference/node/level-demand.qmd
+            - reference/node/flow-boundary.qmd
+            - reference/node/level-boundary.qmd
+            - reference/node/linear-resistance.qmd
+            - reference/node/manning-resistance.qmd
+            - reference/node/fractional-flow.qmd
+            - reference/node/terminal.qmd
+            - reference/node/discrete-control.qmd
+            - reference/node/pid-control.qmd
 
 format:
   html:

docs/background/concept.qmd

@@ -0,0 +1,88 @@
+---
+title: Introduction
+---
+
+# Introduction {#sec-introduction}
+
+Decision makers need to balance the supply and demand of water at the river basin scale, under increasing environmental pressure.
+Ribasim allows users to model basins under current and changing conditions to evaluate and design and management of the water system.
+It is available as free and open source software under the MIT license.
+Besides a model simulation core, Ribasim also includes tooling to assist in building models from basic datasets and visualize results.
+The model and its results provides insights to decision makers, enabling them to build consensus amongst water users and make informed decisions about how to manage water resources optimally.
+
+The model concept of Ribasim is composed of multiple layers:
+- a physical layer representing water bodies and associated infrastructure as well as abstractions,
+- a rule-based control layer to manage the infrastructure, and
+- (optionally) a priority-based allocation layer to take centralized decisions on user abstractions.
+- (optionally) a coupling layer to exchange fluxes and heads with other kernels
+
+Typically hydrological processes on land will be represented in detail by other models which can be coupled (online) to Ribasim with the help of iMOD Coupler.
+Currently, an online coupling with MODFLOW 6 (groundwater) and with Metaswap + MODFLOW 6 (unsaturated zone + groundwater) is available.
+The corresponding documentation can be found within the [iMOD Suite Documentation](https://deltares.github.io/iMOD-Documentation/coupler.html).
+
+This version of Ribasim is the follow up of the legacy Fortran kernel of Ribasim (version 7) applied world wide, the Fortran kernel SIMRES applied in the Netherlands, and the surface water models Distribution Model and Mozart of the Dutch National Hydrological Instrument.
+
+# Concept {#sec-concept}
+
+## Physical layer {#sec-physical}
+
+To represent the physical characteristics of the water system in an area, Ribasim allows you to divide the area into a network of connected representative elementary watersheds ([Reggiani, Sivapalan, and Majid Hassanizadeh 1998](https://deltares.github.io/Ribasim/#ref-REGGIANI1998367)).
+Within Ribasim, these elements are called basins, which are essentially buckets or reservoirs holding an aggregated volume of water bodies in an area.
+Basins are chained in a graph with connector nodes determining the exchange of water between the basins.
+These connector nodes can represent open water connections (e.g. bifurcations or resistance in a free flowing open water channel) or infrastructure elements such as pumps, gates or weirs.
+An overview of node types and associated data inputs is provided on the [usage page](usage.qmd), while the associated mathematical formations are described on the [equations page](equations.qmd).
+
+## Control layer {#sec-control}
+
+Infrastructure elements are often controlled by humans to implement a certain water management strategy.
+Ribasim allows the configuration of conditional rules to influence the exchange of water between basins, either by setting inflow or outflow, or by controlling a water level.
+Control rules evaluate one or multiple conditions to change a parameter setting of an infrastructure element when the conditional criteria are met.
+Conditions can be either calculated values within the network as well as boundary conditions or (todo) external observations, i.e. observation values external to the model.
+An overview of node types and associated data inputs is provided on the [usage page](usage.qmd), while the associated mathematical formations are described on the [equations page](equations.qmd).
+
+## Allocation layer {#sec-allocation}
+
+Ribasim allows water users (water demands) to abstract water from the basins (i.e. from the physical layer) unless the water level drops below a minimum level.
+Under dry conditions, water managers may want to prioritize some abstractions over other abstractions.
+The Ribasim allocation layer can take care of this prioritization by reducing the abstraction rates of lower-priority demands to ensure that sufficient water remains available in the system for the higher-priority demands.
+The associated mathematical formulations are described on the [allocation page](allocation.qmd).
+In case of large networks, a subdivision in a main network with subnetworks is recommended.
+For more details see the explanation of the [simulation loop](core.qmd#sec-nested-allocation).
+
+The layers and the main components and dataflows between the layers are shown in the next figure:
+
+```{mermaid}
+flowchart TB
+physical:::layer
+rbc:::layer
+allocation:::layer
+user_demand
+basin
+connector[basin connector]
+control[control rules]
+condition
+alloc[global allocation]
+
+subgraph physical[physical layer]
+    user_demand-->|abstraction| basin
+    basin<-->|flow| connector
+end
+
+subgraph rbc[rule based control layer]
+   condition --> control
+end
+
+subgraph allocation[allocation layer]
+    alloc
+end
+
+user_demand-->|request demand| alloc
+alloc-->|assign allocation| user_demand
+basin-->|volume| alloc
+basin --> |volume or level| condition
+alloc --> |optional flow update| control
+control --> |action| connector
+
+%% class definitions for C4 model
+classDef layer fill:transparent,stroke-dasharray:5 5
+```

docs/core/index.qmd → docs/background/core.qmd

@@ -5,15 +5,15 @@ title: "Julia core"
 With the term "core", we mean the computational engine of Ribasim. As detailed in the
 [usage](usage.qmd) documentation, it is generally used as a command line tool.
 
-A quick overview of the model concept is available at the [home page](../index.qmd#sec-concept), while a more in depth discussion is available on the [model concept](modelconcept.qmd) page.
+A quick overview of the model concept is available in the [introduction](concept.qmd#sec-concept), while a more in depth discussion is available on the [model concept](modelconcept.qmd) page.
 The theory is described on the [equations](equations.qmd) page, and more in-depth numerical considerations are described on the [numerical considerations](numerics.qmd) page. As allocation is a large and self-contained part of the Ribasim core, it is described on the separate [allocation](allocation.qmd) page. Input validation is described on the [validation](validation.qmd) page.
 
 The core is implemented in the [Julia programming language](https://julialang.org/), and
 can be found in the [Ribasim repository](https://github.com/Deltares/Ribasim) under the
 `core/` folder. For developers we also advise to read the
 [developer documentation](../contribute/core.qmd). Information on coupling can be found [here](coupling.qmd).
 
-An overview of all components is given on the [home page](../index.qmd#sec-components)
+An overview of all components is given in the [Get Started](../tutorials/index.qmd#sec-components) section.
 
 # The simulation loop {#sec-simulationloop}
 

docs/core/modelconcept.qmd → docs/background/modelconcept.qmd

@@ -1,7 +1,7 @@
 ---
 title: Model concept
 ---
-A brief summary of the concept is given on the documentation [home page](../index.qmd#sec-concept).
+A brief summary of the concept is given [introduction](concept.qmd#sec-concept).
 As indicated, the model concept is organized in three layers:
 
 - a physical layer representing water bodies and associated infrastructure,
@@ -110,16 +110,16 @@ In addition to free flowing waterbodies, a watersystem typically has structures
 free flowing conditions or structures.
 The following type of connector nodes are available for this purpose:
 
-- [TabulatedRatingCurve](node/tabulated-rating-curve.qmd): one-directional flow based on upstream head. Node type typically used for gravity flow conditions either free flowing open water channels
+- [TabulatedRatingCurve](../reference/node/tabulated-rating-curve.qmd): one-directional flow based on upstream head. Node type typically used for gravity flow conditions either free flowing open water channels
 or over a fixed structure.
-- [LinearResistance](node/linear-resistance.qmd): bi-directional flow based on head difference and linear resistance. Node type typically used for bi-directional flow
+- [LinearResistance](../reference/node/linear-resistance.qmd): bi-directional flow based on head difference and linear resistance. Node type typically used for bi-directional flow
 situations or situations where head difference over a structure determines its actual flow capacity.
-- [ManningResistance](node/manning-resistance.qmd): bi-directional flow based on head difference and resistance using Manning-Gauckler formula. Same usage as LinearResistance,
+- [ManningResistance](../reference/node/manning-resistance.qmd): bi-directional flow based on head difference and resistance using Manning-Gauckler formula. Same usage as LinearResistance,
 providing a better hydrological meaning to the resistance parameterization.
-- [Pump](node/pump.qmd): one-directional structure with a set flow rate. Node type typically used in combination with control to force water over the edge.
-- [Outlet](node/outlet.qmd): one-directional gravity structure with a set flow rate. Node type typically used in combination with control to force water over the edge, even if
+- [Pump](../reference/node/pump.qmd): one-directional structure with a set flow rate. Node type typically used in combination with control to force water over the edge.
+- [Outlet](../reference/node/outlet.qmd): one-directional gravity structure with a set flow rate. Node type typically used in combination with control to force water over the edge, even if
 their is a mismatch in actual hydraulic capacity. The node type has an automated mechanism to stop the flow when the head difference is zero.
-- [FractionalFlow](node/fractional-flow.qmd): to split an outflow over multiple edges based on a flow fraction. Node type is typically used for diversions or bifurcations with a known and fixed ratio.
+- [FractionalFlow](../reference/node/fractional-flow.qmd): to split an outflow over multiple edges based on a flow fraction. Node type is typically used for diversions or bifurcations with a known and fixed ratio.
 
 The control layer can activate or deactivate nodes, set flow rates for the Pump and Outlet, or choose different parameterizations for TabulatedRatingCurve, LinearResistance, ManningResistance or FractionalFlow.
 

docs/core/usage.qmd → docs/background/usage.qmd

@@ -9,7 +9,7 @@ archive, that must be downloaded and unpacked. It can be placed anywhere, howeve
 important that the contents of the zip file are kept together in a directory. The Ribasim
 CLI executable is in the `bin` directory.
 
-To download `ribasim_cli.zip`, see the [download section](../index.qmd#sec-download).
+To download `ribasim_cli.zip`, see the [download section](../tutorials/index.qmd#sec-download).
 
 To check whether the installation was performed successfully, run `ribasim` with no
 arguments in the command line.

docs/changelog.md

@@ -0,0 +1,32 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
+
+## [Unreleased]
+
+### Added
+- Support for concentration state and time for Delwaq coupling.
+
+### Changed
+- Documentation has been overhauled to be more user-friendly.
+
+
+### Fixed
+
+## [v2024.8.0] - 2024-05-14
+
+### Added
+
+- There is more validation on the edges. #1434
+- If the model does not converge and the used algorithm supports it, we log which Basins don't converge. #1440
+
+### Changed
+
+- If negative storages inadvertently happen, we now throw an error. #1425
+- Users of the QGIS plugin need to remove the old version to avoid two copies due to #1453.
+
+### Fixed
+
+- Performance improvements have been a focus of this release, giving up to 10x faster runs. #1433, #1436, #1438, #1448, #1457
+- The CLI exe is now always in the root of the zip and makes use of the libribasim shared library. #1415

docs/contribute/addnode.qmd → docs/guides/addnode.qmd

@@ -46,7 +46,7 @@ node_id: node ID of the NewNodeType node
 end
 ```
 
-Here `static` refers to data that does not change over time. For naming conventions of these schemas see [Node usage](../core/usage.qmd#sec-node). If a new schema contains a `priority` column for allocation, it must also be added to the list of all such schemas in the function `get_all_priorities` in `util.jl`.
+Here `static` refers to data that does not change over time. For naming conventions of these schemas see [Node usage](../background/usage.qmd#sec-node). If a new schema contains a `priority` column for allocation, it must also be added to the list of all such schemas in the function `get_all_priorities` in `util.jl`.
 
 `validation.jl` deals with checking and applying a specific sorting order for the tabular data (default is sorting by node ID only), see `sort_by_function` and `sorted_table!`.
 
@@ -96,7 +96,7 @@ Non-conservative node types need to be added to the `nonconservative_nodetypes`
 
 ## The Jacobian
 
-See [Equations](../core/equations.qmd#the-jacobian) for a mathematical description of the Jacobian.
+See [Equations](../background/equations.qmd#the-jacobian) for a mathematical description of the Jacobian.
 
 Before the Julia core runs its simulation, the sparsity structure `jac_prototype` of $J$ is determined with `get_jac_prototype` in `sparsity.jl`. This function runs trough all node types and looks for nodes that create dependencies between states. It creates a sparse matrix of zeros and ones, where the ones denote locations of possible non-zeros in $J$. Note that only nodes that set flows in the physical layer (or have their own state like `PidControl`) affect the sparsity structure.
 
@@ -106,7 +106,7 @@ The current dependency groups are:
 
 - Out-neighbor dependencies: examples are `TabulatedRatingCurve`, `Pump` (the latter only in the reduction factor regime and not PID controlled). If the in-neighbor of a node of this group is a basin, then the storage of this basin affects itself and the storage of the outneighbor (or the basin one node further if it is connected with a `FractionalFlow` in between) if that is also a basin;
 - Either-neighbor dependencies: examples are `LinearResistance`, `ManningResistance`. If either the in-neighbor or out-neighbor of a node of this group is a basin, the storage of this basin depends on itself. If both the in-neighbor and the out-neighbor are basins, their storages also depend on eachother.
-- The `PidControl` node is a special case which is discussed in [equations](../core/equations.qmd#sec-PID).
+- The `PidControl` node is a special case which is discussed in [equations](../background/equations.qmd#sec-PID).
 
 Using `jac_prototype` the Jacobian of `water_balance!` is computed automatically using [ForwardDiff.jl](https://juliadiff.org/ForwardDiff.jl/stable/) with memory management provided by [PreallocationTools.jl](https://docs.sciml.ai/PreallocationTools/stable/). These computations make use of `DiffCache` and dual numbers.