-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GetNetworkStats improvment: timeout & caching #1327
GetNetworkStats improvment: timeout & caching #1327
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we can have a better way to handle this heavy calculation process.
The StatsCollector
module is already doing one part of the network stats calculation by requesting only once the stats for all subsets.
We could reuse this module to also calculate only once the network patch for all the GetNetworkStats
message a node receive.
Also those calculation/request are done only by the beacon summary node at summary time (00:00 currenlty) and are usefull before the self repair time (00:05 currently)
So we could use the StatsCollector
module as an orchestrator that will start the 2 task (request/calculation) at summary time (using PubSub event) and delegate the calculation to JobCache
module. Then store all client that request an information and return the result from the JobCache. Then when the self repair time is triggered (using Pubsub event) the module can clean the JobCache as the result will not be relevant anymore.
- subscribe to events to start/stop job cache
- it now use a registry - it might start the process if not already started
Description
The GetNetworkStats message is called by many nodes at the same time, the computation it does is heavy. I added a JobCache so the computation is only done once (cache reset after 30s).
I also added a 5s timer which is bigger than the 3s default.
Fixes #1325
Type of change
How Has This Been Tested?
This can only be tested in stress condition. I'll setup some test environment tomorrow.
Checklist: