Keep Graph
from becoming a bloated junk drawer
#187
Closed
bobmcwhirter
started this conversation in
General
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Per discussion on one of @ctron 's PRs, the
graph
crate is growing large, murky, and ill-defined.I think possibly a structure along the lines of the following might be useful:
Merge the write aspects of
graph
withingestor
and make it theingest
module.This would help solve the problem of how we're testing
graph
currently, using ingestor-y type of things.This module would be hopefully the primary write operational module when it comes to touching the database.
Split the read aspects of
graph
out, to another module,fetch
or similar.But first, determine what form that should really take. We can certainly all share the
entity
crate and its models. We should probably not replicate all of theFooContext
objects that are useful for ensuring a valid walk while writing the graph.Reading will probably be a significantly more-used aspect than writing, at runtime, and so optimizing the fetches makes sense, instead of walking from a found root and aggregating things.
For instance, the
/api/v1/advisory/{sha256}
endpoint currently fetches the advisory, and then walks further/deeper to the joined advisory-vulnerability and vulnerability. This ends up being a 1+N operation, which it does not have to be (https://www.sea-ql.org/SeaORM/docs/relation/data-loader/). Here, the hierarchicFooContext
ofgraph
/ingest
crate is actually limiting towards optimizations.Ultimately, all of this will result in the
graph
crate no longer existing as it does today, and no singular crate namedgraph
at all.Beta Was this translation helpful? Give feedback.
All reactions