Compute mean realized flows #1514

SouthEndMusic · 2024-05-30T19:13:55Z

SouthEndMusic · 2024-05-30T19:15:17Z

Last to do: Add test for mean realized user demand

visr · 2024-06-04T07:37:33Z

core/src/read.jl

+    mean_realized_flows = Dict{Tuple{NodeID, NodeID}, Base.RefValue{Float64}}()
+
+    # Find edges that realize a demand
+    for edge_metadata in values(graph.edge_data)
+        (; type, edge) = edge_metadata
+
+        src_id, dst_id = edge
+        user_demand_inflow = (type == EdgeType.flow) && (dst_id.type == NodeType.UserDemand)
+        level_demand_inflow =
+            (type == EdgeType.control) && (src_id.type == NodeType.LevelDemand)
+        flow_demand_inflow =
+            (type == EdgeType.flow) && has_external_demand(graph, dst_id, :flow_demand)[1]
+
+        if user_demand_inflow || flow_demand_inflow
+            mean_realized_flows[edge] = Ref(0.0)
+        elseif level_demand_inflow
+            mean_realized_flows[(dst_id, dst_id)] = Ref(0.0)
        end


Can you explain why you went for Ref here instead of a Float? Is it for the ergonomics of being able to do

value = dict[key] value[] += x value[] /= dt

value = dict[key] value += x value /= dt dict[key] = value

i.e. having to put it back in? I worry a bit about the performance of this approach.

I got rid of the use of RefValue{Float64}, but now there are more dictionary lookups, e.g.

Ribasim/core/src/callback.jl

Lines 487 to 500 in 8e337aa

for key in keys(mean_input_flows)

mean_input_flows[key] = mean_input_flows[key] / Δt_allocation

end

# Divide by the allocation Δt to obtain the mean realized flows

# from the integrated flows

for (edge, value) in mean_realized_flows

if edge[1] == edge[2]

# Compute the mean realized demand for basins as Δstorage/Δt_allocation

_, basin_idx = id_index(basin.node_id, edge[1])

mean_realized_flows[edge] = value + u[basin_idx]

end

mean_realized_flows[edge] = mean_realized_flows[edge] / Δt_allocation

end

Is there a way to circumvent this?

Dict lookups are quite fast, so I doubt it's a serious issue. I haven't looked into it, but perhaps the compiler can figure out it can optimize this with a single lookup. Also I just tried this, and surprisingly it works. Could be of course that there are still two lookups behind this syntax:

julia> dict = Dict{Int, Int}(1 => 1) Dict{Int64, Int64} with 1 entry: 1 => 1 julia> dict[1] += 1 2 julia> dict Dict{Int64, Int64} with 1 entry: 1 => 2

Fixes #1833 HWS output looks like this before, and no output after. The `invalid_unstable` test model still logs though, as captured by tests. ``` ┌ Info: Convergence bottlenecks in descending order of severity: │ Pump #2019 = NaN │ Pump #2130 = NaN │ Pump #2315 = NaN │ Pump #2637 = NaN │ Pump #3995 = NaN │ Pump #4536 = NaN │ Pump #6128 = NaN │ Pump #6866 = NaN │ Pump #9633 = NaN │ Pump #10704 = NaN │ Pump #10710 = NaN │ Pump #10711 = NaN │ Pump #10712 = NaN │ Pump #10713 = NaN │ Pump #10714 = NaN │ Pump #10715 = NaN │ Pump #10716 = NaN │ Pump #10717 = NaN │ Outlet #6692 = NaN │ Outlet #7679 = NaN │ Outlet #10702 = NaN │ Outlet #10703 = NaN │ UserDemand #1000015 = NaN │ UserDemand #1000016 = NaN │ UserDemand #1000017 = NaN │ UserDemand #1000018 = NaN │ UserDemand #1000019 = NaN │ UserDemand #1000020 = NaN │ UserDemand #1000021 = NaN │ UserDemand #1000022 = NaN │ UserDemand #1000038 = NaN │ UserDemand #1000040 = NaN │ UserDemand #1000042 = NaN │ Basin #529 = NaN │ Basin #670 = NaN │ Basin #905 = NaN │ Basin #1041 = NaN │ Basin #1340 = NaN │ Basin #1369 = NaN │ Basin #1514 = NaN │ Basin #1676 = NaN │ Basin #1709 = NaN │ Basin #1873 = NaN │ Basin #2028 = NaN │ Basin #2232 = NaN │ Basin #2255 = NaN │ Basin #2323 = NaN │ Basin #2336 = NaN │ Basin #2381 = NaN │ Basin #2468 = NaN │ Basin #2496 = NaN │ Basin #2522 = NaN │ Basin #2590 = NaN │ Basin #2749 = NaN │ Basin #2968 = NaN │ Basin #3016 = NaN │ Basin #3090 = NaN │ Basin #3302 = NaN │ Basin #3364 = NaN │ Basin #3448 = NaN │ Basin #3543 = NaN │ Basin #3584 = NaN │ Basin #3721 = NaN │ Basin #3899 = NaN │ Basin #4024 = NaN │ Basin #4134 = NaN │ Basin #4184 = NaN │ Basin #4279 = NaN │ Basin #4339 = NaN │ Basin #4473 = NaN │ Basin #4570 = NaN │ Basin #4714 = NaN │ Basin #4796 = NaN │ Basin #4830 = NaN │ Basin #4838 = NaN │ Basin #5048 = NaN │ Basin #5097 = NaN │ Basin #5207 = NaN │ Basin #5322 = NaN │ Basin #5371 = NaN │ Basin #5381 = NaN │ Basin #5472 = NaN │ Basin #5498 = NaN │ Basin #5505 = NaN │ Basin #5530 = NaN │ Basin #5616 = NaN │ Basin #5909 = NaN │ Basin #5978 = NaN │ Basin #5999 = NaN │ Basin #6017 = NaN │ Basin #6077 = NaN │ Basin #6101 = NaN │ Basin #6164 = NaN │ Basin #6224 = NaN │ Basin #6273 = NaN │ Basin #6327 = NaN │ Basin #6364 = NaN │ Basin #6401 = NaN │ Basin #6419 = NaN │ Basin #6428 = NaN │ Basin #6437 = NaN │ Basin #6473 = NaN │ Basin #6503 = NaN │ Basin #6582 = NaN │ Basin #6757 = NaN │ Basin #6885 = NaN │ Basin #6926 = NaN │ Basin #6945 = NaN │ Basin #6972 = NaN │ Basin #7053 = NaN │ Basin #7087 = NaN │ Basin #7101 = NaN │ Basin #7117 = NaN │ Basin #7164 = NaN │ Basin #7214 = NaN │ Basin #7310 = NaN │ Basin #7385 = NaN │ Basin #7400 = NaN │ Basin #7417 = NaN │ Basin #7484 = NaN │ Basin #7597 = NaN │ Basin #7615 = NaN │ Basin #7668 = NaN │ Basin #7706 = NaN │ Basin #7843 = NaN │ Basin #7909 = NaN │ Basin #8014 = NaN │ Basin #8020 = NaN │ Basin #8049 = NaN │ Basin #8059 = NaN │ Basin #8071 = NaN │ Basin #8234 = NaN │ Basin #8257 = NaN │ Basin #8340 = NaN │ Basin #8364 = NaN │ Basin #8559 = NaN │ Basin #8685 = NaN │ Basin #8717 = NaN │ Basin #8757 = NaN │ Basin #8786 = NaN │ Basin #8817 = NaN │ Basin #8921 = NaN │ Basin #9033 = NaN │ Basin #9086 = NaN │ Basin #9123 = NaN │ Basin #9144 = NaN │ Basin #9185 = NaN │ Basin #9209 = NaN │ Basin #9358 = NaN │ Basin #9406 = NaN │ Basin #9430 = NaN │ Basin #9484 = NaN │ Basin #9586 = NaN │ Basin #9627 = NaN │ Basin #9830 = NaN │ Basin #9885 = NaN │ Basin #9899 = NaN │ Basin #9906 = NaN │ Basin #9954 = NaN │ Basin #10040 = NaN │ Basin #10107 = NaN │ Basin #10184 = NaN │ Basin #10202 = NaN │ Basin #10308 = NaN │ Basin #10321 = NaN │ Basin #10343 = NaN │ Basin #10583 = NaN │ Basin #10587 = NaN │ Basin #10634 = NaN └ Basin #10659 = NaN ```

Compute mean realized flows, fix/add tests

c802e23

Merge branch 'main' into mean_realized_demands

1cd40e3

SouthEndMusic changed the title ~~Compute mean realized flows, fix/add tests~~ Compute mean realized flows Jun 3, 2024

SouthEndMusic added 3 commits June 3, 2024 11:57

Fix tests

f2aa213

Merge branch 'main' into mean_realized_demands

6a49863

Add test for mean realized user demand

8aaebc9

SouthEndMusic requested review from verseve, JoostBuitink and visr and removed request for verseve and JoostBuitink June 3, 2024 11:14

visr reviewed Jun 4, 2024

View reviewed changes

SouthEndMusic added 2 commits June 5, 2024 09:55

Merge branch 'main' into mean_realized_demands

7bf24c2

Don't use RefValue{Float64} as dictionary values

8e337aa

SouthEndMusic requested a review from visr June 5, 2024 08:30

SouthEndMusic added 2 commits June 6, 2024 09:50

Merge branch 'main' into mean_realized_demands

f53c3e1

Small syntax simplifications

dec9b65

visr approved these changes Jun 6, 2024

View reviewed changes

visr merged commit c3d5a40 into main Jun 6, 2024
25 checks passed

visr deleted the mean_realized_demands branch June 6, 2024 08:26

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Compute mean realized flows #1514

Compute mean realized flows #1514

SouthEndMusic commented May 30, 2024

SouthEndMusic commented May 30, 2024

visr Jun 4, 2024

SouthEndMusic Jun 5, 2024

visr Jun 5, 2024

	for key in keys(mean_input_flows)
	mean_input_flows[key] = mean_input_flows[key] / Δt_allocation
	end

	# Divide by the allocation Δt to obtain the mean realized flows
	# from the integrated flows
	for (edge, value) in mean_realized_flows
	if edge[1] == edge[2]
	# Compute the mean realized demand for basins as Δstorage/Δt_allocation
	_, basin_idx = id_index(basin.node_id, edge[1])
	mean_realized_flows[edge] = value + u[basin_idx]
	end
	mean_realized_flows[edge] = mean_realized_flows[edge] / Δt_allocation
	end

Compute mean realized flows #1514

Compute mean realized flows #1514

Conversation

SouthEndMusic commented May 30, 2024

SouthEndMusic commented May 30, 2024

visr Jun 4, 2024

Choose a reason for hiding this comment

SouthEndMusic Jun 5, 2024

Choose a reason for hiding this comment

visr Jun 5, 2024

Choose a reason for hiding this comment