Enforcing aggressive logs and blocks cleanup #745
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…(#12786) * Adding evm_chain_id, address and event_sig to data and topic indexes * Post review fixes * Post review fixes * Post review fixes * Post review fixes * Bumping migration script number --------- Co-authored-by: Domino Valdano <[email protected]>
Current implementation fetches all the CommitReports within the permissionlessExecThreshold from database and filter out result set based on the `snoozedRoots`. It means that when everything is executed, we keep fetching all roots to memory only to discover that we don't need that data. During high traffic spikes, this is an unnecessary waste of resources in the CL node and database as the dataset size grows in time till it reaches permissionlessExec window (8 hours of logs for prod). Example load run in which you can see how it changes over time during steady traffic of messages <img width="1488" alt="image" src="https://github.com/smartcontractkit/ccip/assets/18304098/7b216dd4-1ab3-4943-9797-99a38f4f9eaa"> There are at least two solutions to make it more performant 1. Pass executed roots to the database query to fetch only unexpired roots from the database. Unexpired roots can be determined from the snoozedRoots cache. This is probably the most accurate approach because it always fetches the minimal required set from the database. You can think of the query like this ```sql select * from evm.logs where address = :addr and event_sig = :event_sig and substring(data, X, Y) not in (:executed_roots_array) and block_timestamp > :permissionlessThresholdWindow ``` 2. Make permissionlessThreshold filter "sliding". Commit Reports are executed sequentially for most of the cases, so we can keep track of timestamps of the oldest fully executed report. Instead of inventing a new query for LogPoller we can achieve a lot based on the assumption that execution is sequential. Whenever report is markedAsExecuted we keep bumping block_timestamp filter. We already have that information in cache and indirectly rely on that when filtering out snoozedRoots ```sql and block_timestamp > :oldest_executed_timestamp ``` Solution 2 is easier to implement, doesn't require any changes to LogPoller (and query perf testing) and can be handled in the plugin. We already cache snoozedRoots so we can reuse that to move filtering to the database by narrowing the block_timestamp search window. Assuming that reports are executed sequentially, it should be as accurate as solution 1. duration) <img width="1088" alt="image" src="https://github.com/smartcontractkit/ccip/assets/18304098/c445495d-51e7-4138-a00b-662423fa0cdf">
|
I see you updated files related to core. Please run |
* Adding evm_chain_id, address and event_sig to data and topic indexes smartcontractkit/chainlink#12786 * Improved fetching Commit Reports from database #726
mateusz-sekara
force-pushed
the
aggressive-retention
branch
from
71a248e to
674ad23
Compare
mateusz-sekara
force-pushed
the
aggressive-retention
branch
from
948ace0 to
c56eb31
Compare
mateusz-sekara
force-pushed
the
release/v2.10.0-ccip1.4
branch
from
8624ec6 to
35f0bc5
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Solution