fix: Remove PubSub based PeerDiscovery #1530
Conversation
🦋 Changeset detectedLatest commit: b933b45 The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
|
Removed dependencies detected. Learn more about Socket for GitHub ↗︎ 🚮 Removed packages: @libp2p/[email protected] |
Codecov ReportAttention:
... and 2 files with indirect coverage changes 📢 Thoughts on this report? Let us know!. |
| @@ -1003,6 +1039,12 @@ export class Hub implements HubInterface { | |||
| log.error({ error: syncResult.error, peerId }, "Failed to sync with new peer"); | |||
| } | |||
| } | |||
|
|
|||
| // if the contact info doesn't include a timestamp, consider it invalid but allow the peer to stay in the address book | |||
| // TODO remove this once all peers are updated past 1.6.4 | |||
There was a problem hiding this comment.
Just calling this out as a workaround to still accept peer contacts but to not forward them if they don't include a timestamp.
| @@ -212,7 +220,7 @@ export class LibP2PNode { | |||
| } | |||
| } | |||
| } | |||
| return msgIdFnStrictSign(message); | |||
| return msgIdFnStrictNoSign(message); | |||
There was a problem hiding this comment.
Why the NoSign version? Performance?
There was a problem hiding this comment.
This is misleading. I've pulled this function in from libp2p. The NoSign MsgId just hashes the Message.data field. The Signed MsgId function from earlier instead uses a Key + randomly generated Seq number as the MsgId.
So tldr; This just uses a sha256 hash of the Gossip Message.
Motivation
Hubble instances were running out of Memory due to excessive duplicated traffic on Gossip.
Change Summary
Revoved PubSub based PeerDiscovery in favor of the existing ContactInfos being shared. This reduces amount of network traffic by over 10x while still allowing nodes to be discovered.
The way ContactInfos are added to the Peer Store's AddressBook suffices as a discovery mechism inside of libp2p.
Merge Checklist
Choose all relevant options below by adding an
xnow or at any time before submitting for reviewAdditional Context
The only outstanding consideration with this change is that now, Hubs only publish their own ContactInfos once every 4 hours, or on every new Connection. This means that a new Hub might not see the rest of the network immediately. We could reduce the 4 hour interval to something more reasonable once we test this change out.
PR-Codex overview
This PR focuses on adding a timestamp field to the
ContactInfoContentmessage in thegossip.tsfile.Detailed summary
timestampfield toContactInfoContentmessage ingossip.ts