Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discrete Control performance improvements based on AGV model #1529

Merged
merged 38 commits into from
Jun 12, 2024
Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
5e55795
Use booleans for truth states in the core
SouthEndMusic Jun 3, 2024
4cae47c
Preallocate memory for storing truth state
SouthEndMusic Jun 3, 2024
3cbf958
Refactor control_mappings to avoid (most, not all) runtime dispatch
SouthEndMusic Jun 4, 2024
d04d06d
Fix some tests
SouthEndMusic Jun 4, 2024
c4b2ba2
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 4, 2024
371833a
Fix another test
SouthEndMusic Jun 4, 2024
55bc990
Fix last test
SouthEndMusic Jun 4, 2024
c3a7196
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 4, 2024
9e76ee0
Merge all control mappings into one in discrete_control.control_mapping
SouthEndMusic Jun 5, 2024
e45260b
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 5, 2024
77a2f0d
Cleanup
SouthEndMusic Jun 5, 2024
9a85f22
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 5, 2024
3835d21
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 6, 2024
080e996
Do check on node type in set_control_params! in stead of all the upda…
SouthEndMusic Jun 6, 2024
55e72d6
Some comments adressed
SouthEndMusic Jun 6, 2024
efad525
Comments adressed
SouthEndMusic Jun 6, 2024
7e00275
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 6, 2024
4d35e0a
Small improvements.
evetion Jun 6, 2024
67f04a7
Refactor control_mapping to have a node type specific concretely type…
SouthEndMusic Jun 10, 2024
92c82cc
Remove node type specific methods of `discrete_control_parameter_upda…
SouthEndMusic Jun 10, 2024
68577fe
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 10, 2024
965686d
Add the generic discrete_control_parameter_update! method which magic…
SouthEndMusic Jun 10, 2024
b9f3ee0
Fix tests
SouthEndMusic Jun 10, 2024
ace0b10
Prevent logic_mapping lookup for unchanged truth_state
SouthEndMusic Jun 10, 2024
f86d1f7
Pass tests
SouthEndMusic Jun 11, 2024
01f668c
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 11, 2024
e23bcb9
Merge functions `discrete_control_affect!` and `discrete_control_cond…
SouthEndMusic Jun 11, 2024
0f624fe
Merge branch 'main' into performance_AGV
SouthEndMusic Jun 11, 2024
a2c2d35
Always make discrete_control callback, add docsring to `apply_discret…
SouthEndMusic Jun 11, 2024
5a692eb
Precalculate FlowBoundary outflow ids
SouthEndMusic Jun 11, 2024
db6f4c4
Refactor CompoundVariable to have subvariables
SouthEndMusic Jun 11, 2024
1f06437
Predetermine the flow edges
SouthEndMusic Jun 11, 2024
ee0a77e
Pass tests
SouthEndMusic Jun 11, 2024
90d0bcc
Document metadata of the graph
SouthEndMusic Jun 12, 2024
5e09ff7
Refactor CompoundVariable construction and usage
SouthEndMusic Jun 12, 2024
5ff4b07
Group the compound variables per discrete control node
SouthEndMusic Jun 12, 2024
2e12fe3
Update documentation of `apply_discrete_control!`
SouthEndMusic Jun 12, 2024
35822c2
Small typo fix
SouthEndMusic Jun 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 140 additions & 26 deletions core/src/callback.jl
Original file line number Diff line number Diff line change
Expand Up @@ -289,18 +289,23 @@
# Get the discrete_control node to which this compound variable belongs
discrete_control_node_id = discrete_control.node_id[compound_variable_idx]

# Get the indices of all conditions that this control node listens to
where_node_id = searchsorted(discrete_control.node_id, discrete_control_node_id)

# Get the truth state for this discrete_control node
truth_values = cat(
[
[ifelse(b, "T", "F") for b in discrete_control.condition_value[i]] for
i in where_node_id
]...;
dims = 1,
)
truth_state = join(truth_values, "")
# Get the first index of the (compound) variables that this control node listens to
variable_idx = searchsortedfirst(discrete_control.node_id, discrete_control_node_id)
truth_value_idx = 1

# Build up the truth state from the truth values per (compound) variable per greater than
while variable_idx <= length(discrete_control.node_id) &&
SouthEndMusic marked this conversation as resolved.
Show resolved Hide resolved
discrete_control.node_id[variable_idx] == discrete_control_node_id
truth_values_variable = discrete_control.condition_value[variable_idx]
SouthEndMusic marked this conversation as resolved.
Show resolved Hide resolved
n_greater_than = length(truth_values_variable)
discrete_control.truth_state[truth_value_idx:(truth_value_idx + n_greater_than - 1)] .=
truth_values_variable
variable_idx += 1
truth_value_idx += n_greater_than
end

# The maximum truth state length is possibly shorter than this truth state
truth_state = view(discrete_control.truth_state, 1:(truth_value_idx - 1))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are certain assumptions here on the possible interplays between the different input tables, but I would be really helped to by some inline documentation of such tables.

Both the comment # Build up the truth state from the truth values per (compound) variable per greater than and # The maximum truth state length is possibly shorter than this truth state don't help me much here.

Random thought; we could split such functionality into a function and have a docstring including a doctest?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doctest?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, providing an example (and testing it) as part of the docstring https://documenter.juliadocs.org/stable/man/doctests/.


# What the local control state should be
control_state_new =
Expand All @@ -319,7 +324,7 @@

push!(record.time, integrator.t)
push!(record.control_node_id, Int32(discrete_control_node_id))
push!(record.truth_state, truth_state)
push!(record.truth_state, convert_truth_state(truth_state))
evetion marked this conversation as resolved.
Show resolved Hide resolved
push!(record.control_state, control_state_new)

# Loop over nodes which are under control of this control node
Expand Down Expand Up @@ -393,22 +398,131 @@
return nothing
end

function set_control_params!(p::Parameters, node_id::NodeID, control_state::String)
node = getfield(p, p.graph[node_id].type)
idx = searchsortedfirst(node.node_id, node_id)
new_state = node.control_mapping[(node_id, control_state)]
function update_parameters!(pump::Pump, parameter_update::ParameterUpdate)::Nothing
if parameter_update.node_type != NodeType.Pump
SouthEndMusic marked this conversation as resolved.
Show resolved Hide resolved
return nothing
end
(; node_idx, active, flow_rate_scalar) = parameter_update
pump.active[node_idx] = active
if !isnan(flow_rate_scalar)
SouthEndMusic marked this conversation as resolved.
Show resolved Hide resolved
get_tmp(pump.flow_rate, 0)[node_idx] = clamp(
flow_rate_scalar,
pump.min_flow_rate[node_idx],
pump.max_flow_rate[node_idx],
)
end
return nothing
end

for (field, value) in zip(keys(new_state), new_state)
if !ismissing(value)
vec = get_tmp(getfield(node, field), 0)
vec[idx] = value
end
function update_parameters!(outlet::Outlet, parameter_update::ParameterUpdate)::Nothing
if parameter_update.node_type != NodeType.Outlet
return nothing
end
(; node_idx, active, flow_rate_scalar) = parameter_update
outlet.active[node_idx] = active
if !isnan(flow_rate_scalar)
get_tmp(outlet.flow_rate, 0)[node_idx] = clamp(
flow_rate_scalar,
outlet.min_flow_rate[node_idx],
outlet.max_flow_rate[node_idx],
)
end
return nothing
end

# Set new fractional flow fractions in allocation problem
if is_active(p.allocation) && node isa FractionalFlow && field == :fraction
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this was clearly a red flag in the old code, although I like the genericity of the rest.

set_fractional_flow_in_allocation!(p, node_id, value)
end
function update_parameters!(
tabulated_rating_curve::TabulatedRatingCurve,
parameter_update::ParameterUpdate,
)::Nothing
if parameter_update.node_type != NodeType.TabulatedRatingCurve
return nothing
end
(; node_idx, active, table) = parameter_update
tabulated_rating_curve.active[node_idx] = active
if !isempty(table.t)
tabulated_rating_curve.tables[node_idx] = table
end
return nothing
end

function update_parameters!(
fractional_flow::FractionalFlow,
parameter_update::ParameterUpdate,
p::Parameters,
)::Nothing
if parameter_update.node_type != NodeType.FractionalFlow
return nothing
end
(; node_idx, fraction) = parameter_update
fractional_flow.fraction[node_idx] = fraction

if is_active(p.allocation)
set_fractional_flow_in_allocation!(p, fractional_flow.node_id[node_idx], fraction)
end
return nothing
end

function update_parameters!(
pid_control::PidControl,
parameter_update::ParameterUpdate,
)::Nothing
if parameter_update.node_type != NodeType.PidControl
return nothing
end
(; node_idx, active, target, pid_params) = parameter_update
pid_control.active[node_idx] = active
if !isempty(target.t)
pid_control.target[node_idx] = target
end
if !isempty(pid_params.t)
pid_control.pid_params[node_idx] = pid_params
end
return nothing
end

function update_parameters!(
linear_resistance::LinearResistance,
parameter_update::ParameterUpdate,
)::Nothing
if parameter_update.node_type != NodeType.LinearResistance
return nothing
end
(; node_idx, active, resistance) = parameter_update
linear_resistance.active[node_idx] = active
if !isnan(resistance)
linear_resistance.resistance[node_idx] = resistance
end
return nothing
end

function update_parameters!(
manning_resistance::ManningResistance,
parameter_update::ParameterUpdate,
)::Nothing
if parameter_update.node_type != NodeType.ManningResistance
return nothing
end
(; node_idx, active, manning_n) = parameter_update
manning_resistance.active[node_idx] = active
if !isnan(manning_n)
manning_resistance.manning_n[node_idx] = manning_n

Check warning on line 508 in core/src/callback.jl

View check run for this annotation

Codecov / codecov/patch

core/src/callback.jl#L505-L508

Added lines #L505 - L508 were not covered by tests
end
return nothing

Check warning on line 510 in core/src/callback.jl

View check run for this annotation

Codecov / codecov/patch

core/src/callback.jl#L510

Added line #L510 was not covered by tests
end

function set_control_params!(p::Parameters, node_id::NodeID, control_state::String)::Nothing
parameter_update = p.discrete_control.control_mapping[(node_id, control_state)]

# Only the method corresponding to the type of the node_id actually updates parameters,
# but this way there is no runtime dispatch on node type
update_parameters!(p.pump, parameter_update)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If only we knew the type in parameter_update, so we don't need to call all these functions here ;)

update_parameters!(p.outlet, parameter_update)
evetion marked this conversation as resolved.
Show resolved Hide resolved
update_parameters!(p.tabulated_rating_curve, parameter_update)
update_parameters!(p.fractional_flow, parameter_update, p)
update_parameters!(p.pid_control, parameter_update)
update_parameters!(p.linear_resistance, parameter_update)
update_parameters!(p.manning_resistance, parameter_update)
return nothing
end

function update_subgrid_level!(integrator)::Nothing
Expand Down
45 changes: 36 additions & 9 deletions core/src/parameter.jl
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,29 @@ struct Basin{T, C, V1, V2, V3} <: AbstractParameterNode
time::StructVector{BasinTimeV1, C, Int}
end

"""
ParameterUpdate is an object for storing the data for the
parameter change associated with a specific control state
used by DiscreteControl.
"""
struct ParameterUpdate
node_type::NodeType.T
node_idx::Int
active::Bool
flow_rate_scalar::Float64
flow_rate_itp::ScalarInterpolation
min_crest_level::Float64
resistance::Float64
manning_n::Float64
fraction::Float64
table::ScalarInterpolation
target::ScalarInterpolation
proportional::ScalarInterpolation
integral::ScalarInterpolation
derivative::ScalarInterpolation
pid_params::VectorInterpolation
end

"""
struct TabulatedRatingCurve{C}

Expand Down Expand Up @@ -218,7 +241,7 @@ struct TabulatedRatingCurve{C} <: AbstractParameterNode
active::BitVector
tables::Vector{ScalarInterpolation}
time::StructVector{TabulatedRatingCurveTimeV1, C, Int}
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
end

"""
Expand All @@ -239,7 +262,7 @@ struct LinearResistance <: AbstractParameterNode
active::BitVector
resistance::Vector{Float64}
max_flow_rate::Vector{Float64}
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
end

"""
Expand Down Expand Up @@ -291,7 +314,7 @@ struct ManningResistance <: AbstractParameterNode
profile_slope::Vector{Float64}
upstream_bottom::Vector{Float64}
downstream_bottom::Vector{Float64}
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
end

"""
Expand All @@ -308,7 +331,7 @@ struct FractionalFlow <: AbstractParameterNode
inflow_edge::Vector{EdgeMetadata}
outflow_edge::Vector{EdgeMetadata}
fraction::Vector{Float64}
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
end

"""
Expand Down Expand Up @@ -354,7 +377,7 @@ struct Pump{T} <: AbstractParameterNode
flow_rate::T
min_flow_rate::Vector{Float64}
max_flow_rate::Vector{Float64}
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
is_pid_controlled::BitVector

function Pump(
Expand Down Expand Up @@ -408,7 +431,7 @@ struct Outlet{T} <: AbstractParameterNode
min_flow_rate::Vector{Float64}
max_flow_rate::Vector{Float64}
min_crest_level::Vector{Float64}
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
is_pid_controlled::BitVector

function Outlet(
Expand Down Expand Up @@ -457,6 +480,7 @@ weight: the weight of the variables in the condition per compound variable
look_ahead: the look ahead of variables in the condition in seconds per compound_variable
greater_than: The threshold values per compound variable
condition_value: The current truth value of each condition per compound_variable per greater_than
truth_state: Memory allocated for storing the truth state
control_state: Dictionary: node ID => (control state, control state start)
logic_mapping: Dictionary: (control node ID, truth state) => control state
record: Namedtuple with discrete control information for results
Expand All @@ -470,10 +494,13 @@ struct DiscreteControl <: AbstractParameterNode
look_ahead::Vector{Vector{Float64}}
# Definition of conditions (one or more greater_than per compound variable)
greater_than::Vector{Vector{Float64}}
condition_value::Vector{BitVector}
condition_value::Vector{Vector{Bool}}
# Preallocated for storing truth state
truth_state::Vector{Bool}
# Definition of logic
control_state::Dict{NodeID, Tuple{String, Float64}}
logic_mapping::Dict{Tuple{NodeID, String}, String}
logic_mapping::Dict{Tuple{NodeID, Vector{Bool}}, String}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
record::@NamedTuple{
time::Vector{Float64},
control_node_id::Vector{Int32},
Expand All @@ -500,7 +527,7 @@ struct PidControl{T} <: AbstractParameterNode
target::Vector{ScalarInterpolation}
pid_params::Vector{VectorInterpolation}
error::T
control_mapping::Dict{Tuple{NodeID, String}, NamedTuple}
control_mapping::Dict{Tuple{NodeID, String}, ParameterUpdate}
end

"""
Expand Down
Loading