Meta-issue: Use MetaGraphsNext.jl for graphs with node and edge IDs

[SouthEndMusic]

See here for a motivating discussion.

[visr]

Generalized the issue a bit so it is not only about the allocation graph, but all graphs. And if we can store any metadata on our graphs, we can put node IDs on vertices, edge IDs on edges. Perhaps other metadata like name and node type can also help with simplifying the code, avoiding extra mappings.

https://juliagraphs.org/MetaGraphsNext.jl/stable/tutorial/1_basics/

[SouthEndMusic]

It looks very promising, I think this can really clean up the allocation code and maybe more. What do you think about creating new objects for vertex and edge metadata? Example:

struct VertexMetadata
    type::Symbol # Could replace Parameters.lookup
    data_index::Int # The index of this node in the node data
    state_index::Int # If the node is a basin or PID controller, this is the index of the associated state variable
    subnetwork_id::Int

end

struct EdgeMetadata
    flow::Bool # Is it a flow edge?
    control::Bool # Is it a control edge?
    allocation::Bool # Is it an edge that is part of an allocation graph?
end

It would make sense to me that the vertex labels are the non-consecutive node IDs (although that is discouraged in the link you sent). We also have to think about the construction of the metagraph; I think the metadata must be in some mutable object so that it can be altered after construction. I suppose metadata will not be altered during simulation.

Regarding allocation:

• There is no need to make separate graphs for allocation networks, this can be put in the metadata (note that allocation graphs can have 'shortcut edges' that are not part of the flow graph);
• Thus there is no longer need for mapping between node IDs of allocation graphs and the main flow graph.

[visr]

Yes adding structs for this makes sense to me. I know it's only an example, but to comment on it already:

• Shall we stick to using Node rather than Vertex?
• Instead of a Symbol for type, we could perhaps start using e.g. EnumX.jl instead. That probably also goes for other places in the code.
• Rather than both flow::Bool and control::Bool, here an enum might make more sense as well, so it is always either x or y.

It would make sense to me that the vertex labels are the non-consecutive node IDs (although that is discouraged in the link you sent).

Indeed, this is the mentioned text:

The label_type argument defines how vertices will be referred to, it can be anything you want (although integer types are generally discouraged, to avoid confusion with the vertex codes used by Graphs.jl).

That doesn't mean we shouldn't, but confusing id::Int and index::Int is real. Perhaps we should consider typing our IDs:

struct ID  # or rather, NodeID and EdgeID
    id::Int
end

That means we can dispatch on it, and we don't want to do any arithmetic on ID anyway.

[SouthEndMusic]

Sub-issues for using MetaGraphsNext.jl:

• Combine flow and control graphs using MetaGraphsNext.jl #784: Introduce a single graph graph for flow and control where the edge type is specified in the edge metadata. Here node and edge ID structs can be introduced as label_type and field of the edge_data_type respectively. node type (as a replacement of lookup) and edge type is specified with EnumX.jl.
• Introduce non-nonsecutive node IDs: numbering of nodes #694 (and maybe a number of interrelated issues, it has become a bit messy).
• Refactor allocation.jl with MetaGraphsNext.jl #785: Refactor allocation.jl: Allocation networks are specified in the node and edge metadata. Mapping between node IDs and edge IDs is then no longer required.
• Connectivity refactor #814: Distribute the sparse matrix flow over the metadata of the edges of graph. FYI, the MetaGraph stores the node and edge metadata in dictionaries.
• After this Automatic differentiation w.r.t. time #688 can probably be refactored to be a cleaner fix of autodiff w.r.t. time.

[SouthEndMusic]

Closed as fixed by #789, #807, #828, #830, #850.