Tweaks around libp2p pubsub seen messages cache #3773
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
By default, the libp2p seen messages cache uses the `FirstSeen` strategy which expires an entry once TTL elapses from the time it was added. This means that if a single message is being received frequently and consistently, pubsub will re-broadcast it every TTL, rather than never re-broadcasting it. In the context of the Keep client which additionally uses app-level retransmissions, that often leads to a strong message amplification in the broadcast channel which cause a significant increase in the network load. As the problem is quite common (see libp2p/go-libp2p-pubsub#502), the libp2p team added a new `LastSeen` strategy which behaves differently. This strategy expires an entry once TTL elapses from the last time the message was touched by a cache write (`Add`) or read (`Has`) operation. That gives the desired behavior of never re-broadcasting a message that was already seen within the last TTL period. This reduces the risk of unintended over-amplification.
Once a message is received and validated, pubsub re-broadcasts it to other peers and puts it into the seen messages cache. This way, subsequent arrivals of the same message are not re-broadcasted unnecessarily. This mechanism is important for the network to avoid excessive message flooding. The default value used by libp2p is 2 minutes. However, Keep client messaging sessions are quite time-consuming so, we use a longer TTL of 5 minutes to reduce flooding risk even further. Worth noting that this time cannot be too long as the cache may grow excessively and impact memory consumption.
This was referenced Feb 7, 2024
tomaszslabon
approved these changes
Feb 8, 2024
tomaszslabon
added a commit
that referenced
this pull request
Feb 8, 2024
Closes: #3772 Depends on: #3773 Here we bump up `go-etherum` version to [`v1.13.11`](https://github.com/ethereum/go-ethereum/releases/tag/v1.13.11). This version is ready for the Cancun-Deneb (Dencun) upgrade and adds support for the new “blob-carrying” EIP-4844 transaction type in API methods. The RPC client used in older `go-ethereum` versions does not recognize EIP-4844 transactions and may error out if there is a need to parse return data from transaction-related functions called against chains where the Dencun upgrade has been enabled. We observed this problem on our Sepolia testnet when calling `eth_getBlockByNumber` using version `v1.10.19` after Jan 31th so after the date when Dencun was enabled on Sepolia We are also taking an opportunity and optimize our `GetBlockHashByNumber` function. So far this function called the `BlockByNumber` function of the RPC client. Under the hood, that resulted in an inefficient `eth_getBlockByNumber` call with the `show transaction details` flag set to `true` which loaded full transaction data of the given block which is not necessary for the context of `GetBlockHashByNumber`. Here we improve that behavior by using the `HeaderByNumber` function of the RPC client. That function calls `eth_getBlockByNumber` with the `show transaction details` flag set to `false` which does not load transaction data and returns only data specific to the block header. This is enough to get the hash of the block.
lukasz-zimnoch
added a commit
that referenced
this pull request
Feb 12, 2024
This pull request backports #3773 to the `releases/mainnet/v2.0.0-m7` branch.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs: #3770
Depends on: #3771
Recent libp2p versions (we started to use them in #3771) introduced a way to set the seen messages cache TTL and strategy. Here we leverage those settings to reduce the excessive message flooding effect that sometimes occurs on mainnet. This pull request consists of two steps
Use longer TTL for pubsub seen messages cache
Once a message is received and validated, pubsub re-broadcasts it to other peers and puts it into the seen messages cache. This way, subsequent arrivals of the same message are not re-broadcasted unnecessarily. This mechanism is important for the network to avoid excessive message flooding. The default value used by libp2p is 2 minutes. However, Keep client messaging sessions are quite time-consuming so, we use a longer TTL of 5 minutes to reduce flooding risk even further. Worth noting that this time cannot be too long as the cache may grow excessively and impact memory consumption.
Use
LastSeenas seen messages cache strategyBy default, the libp2p seen messages cache uses the
FirstSeenstrategy which expires an entry once TTL elapses from when it was added. This means that if a single message is being received frequently and consistently, pubsub will re-broadcast it every TTL, rather than never re-broadcasting it.In the context of the Keep client which additionally uses app-level retransmissions, that often leads to a strong message amplification in the broadcast channel which causes a significant increase in the network load.
As the problem is quite common (see libp2p/go-libp2p-pubsub#502), the libp2p team added a new
LastSeenstrategy which behaves differently. This strategy expires an entry once TTL elapses from the last time the message was touched by a cache write (Add) or read (Has) operation. That gives the desired behavior of never re-broadcasting a message that was already seen within the last TTL period. This reduces the risk of unintended over-amplification.