Update Neo4j Library

[TheTechromancer]

Since py2neo has been deprecated, we need to transition our Neo4j output module to use the official Neo4j python library instead.

In the meantime, a temporary fix for py2neo has been issued here: #773.

[stryker2k2]

Doing research on using neomodel instead of py2neo, as suggested by the developers of Neo4j in their Py2neo Is End-of-Life Blog

[stryker2k2]

We will be using the native neo4j library utilizing the neo4j Driver instead of neomodel or py2neo.

I have pushed a branch that successfully creates a Node and a Relationship for each event using the Neo4j Driver.

At this moment, I am having issues with some Relationships being successfully established but not visibly linked when looking at the Neo4j output. The debug output shows the same... the relationship query is a success but 0 relationships are established sometimes.

[stryker2k2]

We successfully moved away from py2neo and we are using the Official Neo4j "driver".

I was able to easily get the Nodes to populate... but I had some difficulty getting the relationships to connect to Source Nodes that had not yet been created.

I have queued those Relationships that did not have an existing Source Node for later parsing. Once the Source Node becomes available, that relationship is now established.

I have pushed this update_neo4j branch and I will polish it up before making a Merge Request (Pull Request).

TODO: Run against a big target with all the modules activated. Troubleshoot as needed.

[stryker2k2]

Tinkering around with the idea of sending a "MATCH (n) DETACH DELETE (n)" command before every new BBOT Scan.

[stryker2k2]

Tinkering around with the idea of sending a "MATCH (n) DETACH DELETE (n)" command before every new BBOT Scan.

I have implemented this functionality which now removes all previous Neo4j data. As of right now, it ALWAYS clears the previous Neo4j data... it DOES NOT ASK or accept command prompt args.

[stryker2k2]

It works. But, something isn't quite working as intended. It's not broke... just... not what I expected to see.

[stryker2k2]

I re-ran the dev branch with -om neo4j (py2neo). The output looks a lot similar to my update_neo4j branch now. All that is left to do is to make relationships between CODE_REPOSITORY have a relationship with the ORG_STUB that SPECULATE emitted.

[stryker2k2]

It just keeps getting better and better! I fixed an error that was occuring when ASN was being parsed where the quotations would confuse Neo4j and it would throw an error.

Furthermore, I have made a single (post-init) driver.session for our output module to utiliize for all events and we now properly clean up after ourselves.

[TheTechromancer]

Implemented in #1205.