Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add fair distribution options #751

Merged
merged 35 commits into from
Nov 20, 2023
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
d7af65a
Add allocation output
SouthEndMusic Nov 6, 2023
c043d16
Merge branch 'main' into allocation_output_file
SouthEndMusic Nov 6, 2023
b0c549a
Add test
SouthEndMusic Nov 7, 2023
061e6f8
Merge branch 'main' into allocation_output_file
SouthEndMusic Nov 7, 2023
442a52b
Tests completed
SouthEndMusic Nov 7, 2023
9f3033e
Test adjusted
SouthEndMusic Nov 7, 2023
4d7123c
Add fair distribution options
SouthEndMusic Nov 7, 2023
3554b53
Towards fair distribution options
SouthEndMusic Nov 8, 2023
9e27e9c
Merge branch 'main' into allocation_output_file
SouthEndMusic Nov 8, 2023
c7115c5
Let allocation default to allocated = demand
SouthEndMusic Nov 8, 2023
f827910
Merge branch 'main' into allocation_output_file
SouthEndMusic Nov 8, 2023
6e3252c
Take minimum of current demand and optimized allocation
SouthEndMusic Nov 8, 2023
bcfc493
docs update
SouthEndMusic Nov 8, 2023
70484e1
Merge branch 'main' into fair_distribution
SouthEndMusic Nov 8, 2023
558e421
Use JuMP.@objective
SouthEndMusic Nov 8, 2023
e572e9e
Merge branch 'allocation_output_file' into fair_distribution
SouthEndMusic Nov 8, 2023
8a8c63b
Towards testing all objective types
SouthEndMusic Nov 10, 2023
c35399b
Merge branch 'main' into fair_distribution
SouthEndMusic Nov 13, 2023
dc1c09f
Manifest fix
SouthEndMusic Nov 13, 2023
29c2b09
Make all allocation objective functions run without error
SouthEndMusic Nov 14, 2023
87e9d0b
Merge branch 'main' into fair_distribution
SouthEndMusic Nov 14, 2023
d51fee7
Add docstrings
SouthEndMusic Nov 14, 2023
9467f2e
Add tests
SouthEndMusic Nov 14, 2023
186dcc7
Some more tests
SouthEndMusic Nov 14, 2023
132660e
Docs update
SouthEndMusic Nov 14, 2023
62480c2
division by 0 clarification
SouthEndMusic Nov 14, 2023
71dc2c0
Absolute value clarification
SouthEndMusic Nov 14, 2023
c81afe9
Merge branch 'main' into fair_distribution
SouthEndMusic Nov 16, 2023
6289c0e
Merge branch 'main' into fair_distribution
visr Nov 16, 2023
f76ddca
Merge branch 'main' into fair_distribution
SouthEndMusic Nov 17, 2023
8a96333
Merge branch 'fair_distribution' of https://github.com/Deltares/Ribas…
SouthEndMusic Nov 17, 2023
11baf16
Comments adressed
SouthEndMusic Nov 17, 2023
b6237b1
Merge branch 'main' into fair_distribution
SouthEndMusic Nov 20, 2023
4071b07
Run Julia tests in parallel (#803)
visr Nov 20, 2023
fd767a3
Merge branch 'main' into fair_distribution
visr Nov 20, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion core/src/Ribasim.jl
Original file line number Diff line number Diff line change
Expand Up @@ -61,9 +61,9 @@ TimerOutputs.complement!()

include("validation.jl")
include("solve.jl")
include("allocation.jl")
include("config.jl")
using .config
include("allocation.jl")
include("utils.jl")
include("lib.jl")
include("io.jl")
Expand Down
175 changes: 122 additions & 53 deletions core/src/allocation.jl
Original file line number Diff line number Diff line change
Expand Up @@ -387,34 +387,20 @@ function add_variables_flow!(
end

"""
Add the basin allocation variables A_basin to the allocation problem.
The variable indices are the allocation graph basin node IDs.
Non-negativivity constraints are also immediately added to the basin allocation variables.
Certain allocation distribution types use absolute values in the objective function.
Since most optimization packages do not support the absolute value function directly,
New variables are introduced that act as the absolute value of an expression by
posing the appropriate constraints.
"""
function add_variables_allocation_basin!(
problem::JuMP.Model,
node_id_mapping::Dict{Int, Tuple{Int, Symbol}},
allocgraph_node_ids_basin::Vector{Int},
)::Nothing
JuMP.@variable(problem, A_basin[i = allocgraph_node_ids_basin] >= 0.0)
return nothing
end

"""
Add the user allocation constraints to the allocation problem:
The flow to a user is bounded from above by the demand of the user.
"""
function add_constraints_user_allocation!(
function add_variables_absolute_value!(
problem::JuMP.Model,
allocgraph_edge_ids_user_demand::Dict{Int, Int},
config::Config,
)::Nothing
F = problem[:F]
# Allocation flows are bounded from above by demands
problem[:demand_user] = JuMP.@constraint(
problem,
[i = values(allocgraph_edge_ids_user_demand)],
F[i] <= 0.0
)
if startswith(config.allocation.objective_type, "linear")
problem[:F_abs] =
JuMP.@variable(problem, F_abs[values(allocgraph_edge_ids_user_demand)])
end
return nothing
end

Expand Down Expand Up @@ -549,27 +535,63 @@ function add_constraints_user_returnflow!(
end

"""
Add the objective function to be maximized to the allocation problem.
Objective function: Sum of flows to the users.
Minimizing |expr| can be achieved by introducing a new variable expr_abs
and posing the following constraints:
expr_abs >= expr
expr_abs >= -expr
"""
function add_objective_function!(
function add_constraints_absolute_value!(
problem::JuMP.Model,
allocgraph_edge_ids_user_demand::Dict{Int, Int},
config::Config,
)::Nothing
F = problem[:F]
A_basin = problem[:A_basin]
JuMP.@objective(
problem,
Max,
sum(A_basin) + sum([F[i] for i in values(allocgraph_edge_ids_user_demand)])
)
objective_type = config.allocation.objective_type
if startswith(objective_type, "linear")
allocgraph_edge_ids_user_demand = collect(values(allocgraph_edge_ids_user_demand))
F = problem[:F]
F_abs = problem[:F_abs]
d = 2.0

if config.allocation.objective_type == "linear_absolute"
# These constraints together make sure that F_abs acts as the absolute
# value F_abs = |x| where x = F-d (here for example d = 2)
problem[:abs_positive] = JuMP.@constraint(
problem,
[i = allocgraph_edge_ids_user_demand],
F_abs[i] >= (F[i] - d),
base_name = "abs_positive"
)
problem[:abs_negative] = JuMP.@constraint(
problem,
[i = allocgraph_edge_ids_user_demand],
F_abs[i] >= -(F[i] - d),
base_name = "abs_negative"
)
elseif config.allocation.objective_type == "linear_relative"
# These constraints together make sure that F_abs acts as the absolute
# value F_abs = |x| where x = 1-F/d (here for example d = 2)
problem[:abs_positive] = JuMP.@constraint(
problem,
[i = allocgraph_edge_ids_user_demand],
F_abs[i] >= (1 - F[i] / d),
base_name = "abs_positive"
)
problem[:abs_negative] = JuMP.@constraint(
problem,
[i = allocgraph_edge_ids_user_demand],
F_abs[i] >= -(1 - F[i] / d),
base_name = "abs_negative"
)
end
end
return nothing
end

"""
Construct the allocation problem for the current subnetwork as a JuMP.jl model.
"""
function allocation_problem(
config::Config,
node_id_mapping::Dict{Int, Tuple{Int, Symbol}},
allocgraph_node_ids_user_with_returnflow::Vector{Int},
allocgraph_edges::Vector{Edge{Int}},
Expand All @@ -591,11 +613,10 @@ function allocation_problem(

# Add variables to problem
add_variables_flow!(problem, allocgraph_edges)
add_variables_allocation_basin!(problem, node_id_mapping, allocgraph_node_ids_basin)
add_variables_absolute_value!(problem, allocgraph_edge_ids_user_demand, config)
# TODO: Add variables for allocation to basins

# Add constraints to problem
add_constraints_user_allocation!(problem, allocgraph_edge_ids_user_demand)
add_constraints_basin_allocation!(problem, allocgraph_node_ids_basin)
add_constraints_capacity!(problem, capacity, allocgraph_edges)
add_constraints_source!(
problem,
Expand All @@ -610,11 +631,9 @@ function allocation_problem(
allocgraph_node_outedge_ids,
)
add_constraints_user_returnflow!(problem, allocgraph_node_ids_user_with_returnflow)
add_constraints_absolute_value!(problem, allocgraph_edge_ids_user_demand, config)
# TODO: The fractional flow constraints

# Add objective to problem
add_objective_function!(problem, allocgraph_edge_ids_user_demand)

return problem
end

Expand All @@ -640,6 +659,7 @@ An AllocationModel object.

"""
function AllocationModel(
config::Config,
allocation_network_id::Int,
p::Parameters,
subnetwork_node_ids::Vector{Int},
Expand All @@ -658,6 +678,7 @@ function AllocationModel(

# The JuMP.jl allocation problem
problem = allocation_problem(
config,
node_id_mapping,
allocgraph_node_ids_user_with_returnflow,
allocgraph_edges,
Expand All @@ -668,6 +689,7 @@ function AllocationModel(
)

return AllocationModel(
Symbol(config.allocation.objective_type),
allocation_network_id,
subnetwork_node_ids,
node_id_mapping,
Expand All @@ -682,27 +704,69 @@ function AllocationModel(
end

"""
Set the demands of the users of the current time and priority
in the allocation problem.
Set the objective for the given priority.
For an objective with absolute values this also involves adjusting constraints.
"""
function set_demands_priority!(
function set_objective_priority!(
allocation_model::AllocationModel,
user::User,
priority_idx::Int,
t::Float64,
priority_idx::Int,
)::Nothing
(; problem, allocgraph_edge_ids_user_demand, node_id_mapping_inverse) = allocation_model
(; objective_type, problem, allocgraph_edge_ids_user_demand, node_id_mapping_inverse) =
allocation_model
(; demand, node_id) = user
constraints_demand = problem[:demand_user]
F = problem[:F]
if objective_type in [:quadratic_absolute, :quadratic_relative]
ex = JuMP.QuadExpr()
elseif objective_type in [:linear_absolute, :linear_relative]
ex = sum(problem[:F_abs])
end

for (allocgraph_node_id, allocgraph_edge_id) in allocgraph_edge_ids_user_demand
model_user_id = node_id_mapping_inverse[allocgraph_node_id][1]
user_idx = findsorted(node_id, model_user_id)
JuMP.set_normalized_rhs(
constraints_demand[allocgraph_edge_id],
demand[user_idx][priority_idx](t),
)
user_idx = findsorted(node_id, node_id_mapping_inverse[allocgraph_node_id][1])
d = demand[user_idx][priority_idx](t)

if objective_type == :quadratic_absolute
# Objective function ∑ (F - d)^2
F_ij = F[allocgraph_edge_id]
JuMP.add_to_expression!(ex, 1, F_ij, F_ij)
JuMP.add_to_expression!(ex, -2 * d, F_ij)
JuMP.add_to_expression!(ex, d^2)

elseif objective_type == :quadratic_relative
# Objective function ∑ (1 - F/d)^2S
if d ≈ 0
continue
end
F_ij = F[allocgraph_edge_id]
JuMP.add_to_expression!(ex, 1.0 / d^2, F_ij, F_ij)
JuMP.add_to_expression!(ex, -2.0 / d, F_ij)
JuMP.add_to_expression!(ex, 1.0)

elseif objective_type == :linear_absolute
# Objective function ∑ |F - d|
JuMP.set_normalized_rhs(problem[:abs_positive][allocgraph_edge_id], -d)
JuMP.set_normalized_rhs(problem[:abs_negative][allocgraph_edge_id], d)

elseif objective_type == :linear_relative
# Objective function ∑ |1 - F/d|
JuMP.set_normalized_coefficient(
problem[:abs_positive][allocgraph_edge_id],
F[allocgraph_edge_id],
iszero(d) ? 0 : 1 / d,
)
JuMP.set_normalized_coefficient(
problem[:abs_negative][allocgraph_edge_id],
F[allocgraph_edge_id],
iszero(d) ? 0 : -1 / d,
)
else
error("Invalid allocation objective type $objective_type.")
end
end
new_objective = JuMP.@expression(problem, ex)
JuMP.@objective(problem, Min, new_objective)
return nothing
end

Expand Down Expand Up @@ -818,7 +882,12 @@ function allocate!(p::Parameters, allocation_model::AllocationModel, t::Float64)
# or set edge capacities if priority_idx = 1
adjust_edge_capacities!(allocation_model, priority_idx)

set_demands_priority!(allocation_model, user, priority_idx, t)
# Set the objective depending on the demands
# A new objective function is set instead of modifying the coefficients
# of an existing objective function because this is not supported for
# quadratic terms:
# https://jump.dev/JuMP.jl/v1.16/manual/objective/#Modify-an-objective-coefficient
set_objective_priority!(allocation_model, user, t, priority_idx)

# Solve the allocation problem for this priority
JuMP.optimize!(problem)
Expand Down
1 change: 1 addition & 0 deletions core/src/config.jl
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ end
@option struct Allocation <: TableOption
timestep::Union{Float64, Nothing} = nothing
use_allocation::Bool = false
objective_type::String = "quadratic_relative"
end

@option @addnodetypes struct Config <: TableOption
Expand Down
13 changes: 13 additions & 0 deletions core/src/create.jl
Original file line number Diff line number Diff line change
Expand Up @@ -277,6 +277,7 @@ function generate_allocation_models!(p::Parameters, db::DB, config::Config)::Not
push!(
connectivity.allocation_models,
AllocationModel(
config,
allocation_network_id,
p,
allocation_group_node.fid,
Expand Down Expand Up @@ -786,6 +787,18 @@ function User(db::DB, config::Config)::User
abstracted = Float64[],
)

record = (
time = Vector{Float64}(),
allocation_network_id = Vector{Int}(),
user_node_id = Vector{Int}(),
priority = Vector{Int}(),
demand = Vector{Float64}(),
allocated = Vector{Float64}(),
abstracted = Vector{Float64}(),
)

allocation_optimized = BitVector(zeros(UInt8, length(node_ids)))

return User(
node_ids,
active,
Expand Down
2 changes: 2 additions & 0 deletions core/src/solve.jl
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ const VectorInterpolation =
"""
Store information for a subnetwork used for allocation.

objective_type: The name of the type of objective used
allocation_network_id: The ID of this allocation network
node_id: All the IDs of the nodes that are in this subnetwork
node_id_mapping: Mapping Dictionary; model_node_id => AG_node_id where such a correspondence exists
Expand All @@ -20,6 +21,7 @@ problem: The JuMP.jl model for solving the allocation problem
Δt_allocation: The time interval between consecutive allocation solves
"""
struct AllocationModel
objective_type::Symbol
allocation_network_id::Int
node_id::Vector{Int}
node_id_mapping::Dict{Int, Tuple{Int, Symbol}}
Expand Down
Loading